[coyotos-dev] Sleep, wakeup, and persistence

[Sam Mason]

On Fri, Sep 14, 2007 at 11:00:38PM -0400, Jonathan S. Shapiro wrote:
> On Fri, 2007-09-14 at 20:49 +0100, Sam Mason wrote:
> > On Fri, Sep 14, 2007 at 02:03:51PM -0400, Jonathan S. Shapiro wrote:
> > > I want to document a discussion JWA and I had yesterday. This solution
> > > is probably well known; it just wasn't well known to us.
> > 
> > Our last discussion of time[1] had the kernel stripped of almost all
> > time related functionality.  Is that still the intention, or have things
> > changed?
> 
> This was my original intention, but it isn't going to work. Drivers that
> are outside of the kernel need to be able to sleep for very short
> intervals. The addition of a user-mode agent in these sleeps can in some
> cases introduce prohibitive overhead.

I'd agree, for most sleeps a simple kernel based sleep is going to
be enough and any user-level service is going to introduce much more
overhead than it could usefully pay back in any other way.

> BUT:
> 
> It doesn't really matter. Whether there is one sleeping application or
> many actually does not change the required kernel interface.

I'd agree as well.

> In the presence of checkpointing, I believe that the only correct
> handling of relative sleeps (sleep for K units of time) is to turn them
> into absolute deadlines. To enumerate the possibilities, we have:
> 
>   1. Cancel all such sleeps on restart.
>   2. Convert these sleeps to absolute deadlines so that we know
>      which ones have not ended yet at the time of restart.
>   3. Allow the sleeps to replay, with the effect that the sleep
>      may run for a much longer time than intended.
> 
> In the driver case, the driver *must* know that a restart has occurred,
> because various hardware state machines have been reset. If the drivers
> are non-persistent, all is well, but for persistent drivers there is a
> serious design challenge here.

It seems to me that the only sensible thing the kernel can do is the
first option, that of cancelling all sleeps upon restart.  If there's
a chance that the kernel isn't going to know the time when it restarts
then it can't do anything else.

Just to throw another (probably redundant) point in, are the drivers
going to be anything like I remember reading about (somewhere!)
previously: drivers wouldn't have persistent capabilities to the
hardware and would hence know that a restart had occurred because their
capability was severed at restart.  If sleeps were cancelled at restart
it would make recovery easier as the driver wouldn't have to worry about
getting responses from sleeps at inconvenient times later on, but I
think that's minor.

If applications want to wait until a specific time, they will have
needed to have got the current time from somewhere.  When their sleep is
cancelled they can reacquire the current time and resume their sleep.

> > There seem to be two use cases to me; that of sleeping for a short
> > amount of time, and that of sleeping until we reach some specific real
> > time.  Why not put the first in the kernel, and leave the second up to
> > userspace to implement.
> 
> The most natural implementation of "sleep until absolute time" is to
> convert into "sleep until absolute system time unit". 

If you want to support this, then yes I'd agree.

> Unfortunately, the
> kernel probably *does* need to know how to convert intervals of the form
> "k microseconds" into intervals of the form "k fundamental time units".

Yes, and that's not a problem.  The problem, for me, occurs when you try
and migrate these time units between restarts.

> It was never my intention that the kernel should understand absolute
> time in human terms. It was only my intention that the kernel should
> understand absolute time in terms of fundamental system time units in
> order to be able to establish a sort on deadlines, and also in order to
> determine whether sleep cancellation is required.

I agree, that seems like the best thing to do.


System startup is going to be a VERY busy time if everything gets
notified that all their sleeps have been cancelled!  I doubt that this
is going to be more work than currently happens when a (non-persistent)
system boots.


  Sam