[coyotos-dev] Need for revised invocation

[Jonathan S. Shapiro]

On Mon, 2006-02-06 at 10:54 -0800, Charles Landau wrote:
> As I've said, I think a kernel-implemented helper domain could set 
> such a global variable. I'm not yet convinced that all the options 
> for avoiding user-level scheduling have been exhausted. If they have, 
> it should be possible to construct a proof that the requirements 
> cannot be met otherwise.

Why should I have to spend an entire kernel thread (4k register state,
4k stack) to set a single word to 1?

In any case, the problem for the TCP slow timeout is a little different
from the problem for video decoding. In the TCP slow timeout case, you
actually aren't going to preempt the main stream of processing. A
surprisingly large amount of scheduling fuzz is acceptable when getting
that word set. In the video case, the tolerable fuzz is much smaller,
and in the audio case, it is VERY much below the reasonable quanta
interval and MUST be delivered preemptively. By "preemptively", I mean
that all acceptable solutions involve some form of shared-memory
concurrency. At that point, we are reduced to the "kernel thread or user
thread" argument, and we are arguing about mechanisms rather than dogma.


> The big issue is, who is in charge of allocating the CPU resource? It 
> is simplest to centralize this function (in the kernel). Of course 
> simplest isn't always adequate. If you believe it is necessary to 
> migrate some of this function down into user code (and I'm still not 
> convinced), you'll have to impart some knowledge of global scheduling 
> conditions into the user code. For example, the multimedia event 
> handlers might not want to abandon their work if there is an idle CPU 
> available. I'm quite sure you've thought about these issues, but I 
> haven't seen a credible design yet. (I haven't yet reviewed the 
> recent ukernel spec.)

Actually, I have not invested a great deal of work in investigating
this. The arguments in favor of exposing the kernel-level scheduler
actions to the user-level thread scheduler have been pretty thoroughly
explored in the literature. There is a consensus among people building
multimedia systems that this type of exposure is essential for their
applications.

In EROS, I might have tried to argue that a purer approach was better. I
would have suffered in this argument because I don't know the issues in
multimedia programming well enough to prevail in the argument.

Ultimately, however, any victory would be a pyrhic victory. The
consequence of a successful argument for purity would be that multimedia
application builders would simply not target Coyotos.


Much the same is true for a bunch of other applications. My sense at the
moment is this:

There are parts of a system that really need to be robust. I agree with
you that these should generally be single-threaded -- or possibly in
some cases event driven. There *are* safe idioms here for concurrency
and overlapping execution, but they are intentionally restricted idioms.
Some of these idioms were contemplated in KeyKOS. For example: page
read-ahead. Similarly, there are restricted safe idioms for preemption.

Separately, there are parts of a system that need to be easily ported.
Almost by definition, such parts are NOT robust. When you start talking
about porting existing crap code there is really no getting away from
the crap. Examples in this category include movie and audio players and
POSIX programs. The programming idioms used in these programs may or may
not be ill-advised. The users have declared that they do not care.

My sense is that a middle ground for the non-robust programs can be had
by robust confinement. This leaves us with questions of performance and
scalability for these ill-advised idioms.

Given enough kernel threads, you can make POSIX select work just fine.
You can make preemption work through kernel multithreading (though there
are upcall problems that appear to need *some* kernel assist when trust
is asymmetric). The problem with the kernel multi-threading approach is
that the space requirements do not scale. A second problem is that given
three objectives:

	A resume-only design (no logical preemption)
	A blocking IPC
	An unbuffered IPC

there appear to be legitimate, reasonable, and robust design patterns
that require "choose two of three". We chose to experiment with relaxing
the blocking constraint.

Given that we need to add *something* to the kernel to solve these
problems, I tend to prefer FCRBs to the kernel spin-agent approach (the
one you proposed). FCRBs move existing complexity from one place to
another and solve a whole bunch of these concerns at once. The spin
agent doesn't.


shap