[coyotos-dev] Thoughts on FCRBs and asynchrony

[Marcus Brinkmann]

At Tue, 7 Feb 2006 15:20:08 -0700,
"Christopher Nelson" <paradox at bbhc.org> wrote:
> My first question is, how do activations (or continuations) simplify a
> program versus threads, when you consider that an activation (or
> continuation) is basically just a special thread?

Activations don't do away with basic problems of concurrent
programming, like the need for proper locking of critical regions.
However, they do make it possible to implement various techniques like
preemption, M-to-N thread models, multiple outstanding RPCs etc which
are difficult or expensive in a synchronous kernel.

> My second question is regarding kernel threads vs. user threads.  I
> don't understand what the differentiation is between them, that makes
> user threads so much "lighter" than kernel threads.  For example, in my
> kernel there is one TSS per CPU, so I don't have that overhead.  But I
> still have to save all processor state.  Whether I do that in the kernel
> or in userspace doesn't matter.  I still have to save them, so I still
> have the same space overhead.  In order to preemptively reschedule them,
> I still have to do a ring transition.  Whether that calls a kernel mode
> scheduler or a user mode scheduler, you still have that overhead.  The
> only way that makes any sort of difference is when you know that user
> mode thread can't make any more progress, and it voluntarily gives up
> it's execution time to some other thread.  In that case you have no ring
> transitions.

Just because some specific implementations or uses of activations to
implement user-level threads do not allow optimizations doesn't mean
that optimizations are never possible.  If you actually have N
concurrent user level threads which need all their state to be
preserved, then the cost of the state for the N threads is a cost that
you, as the application programmer, are willing to accept!

However, as you said yourself, there are cases where, with
activations, certain optimizations are possible.  It is for those
cases were the difference matters.

Let me give you a very specific example.  In a multi-server object
system based on a synchronous kernel like EROS or L4, implementing the
select() POSIX call requires, in the worst case, as many threads as
there are file descriptors, because you need to do that many
concurrent blocking RPC operations (in fact, if the client is
oblivious of the server implementing the object, as is the case in
session-less protected object systems, every case is the worst case).

These "helper threads" serve no function except to allow the client to
perform concurrent calls.  They are set up for this purpose, and
immediately destroyed afterwards.  They don't have any interesting
state except for the result of the RPC.  It should be clear that in
this case, you can optimize everything away except for the FCRB.

I think similar arguments can be made for other use cases.

> Now, if you're talking continuations there may be some space and time
> optimizations.  But really, you still have to store and load registers
> to resume a continuation.  Plus, if you plan on hopping back and forth,
> they both need a separate stack.  The only real space savings you get is
> if one continuation executes, finishes, and then resumes the previous
> continuation.  Or if the other continuation stores all of it's important
> state on the heap so that it can resume from "scratch" cpu state each
> time, but still start up where it left off job-wise.

The activation handler should run with a small register set.  If you
are talking about real preemption of continuations (ie to run another
continuation), then you are talking about real concurrency and thus
about a model with multiple (real) user threads.  This seems confusing
the issues to me.

Thanks,
Marcus