[coyotos-dev] First Class Activations, Second Class IPC: a UNIXoid and inaccurate summary

Dominique Quatravaux dom at kilimandjaro.dyndns.org
Fri Jan 20 05:45:11 EST 2006

Here is an attempt of a layman (by this list's standards) to summarize
what I understand on the issue, with a heavy dose of UNIX intellectual
taint. Please reply and correct my misunderstandings. I changed the
title from "First Class Messages" to "First Class Activations", because
I believe that the term "message" is now misleading as it encompasses
two different concepts (see the bottom of my post).

>The FCRB design is trying to address the
>interaction of two requirements:
>  1. The need for preemptive event delivery
>  2. The need for non-blocking sends, which implies the need for
>     non-preemptible receives.
So the point of the discussion is to revise the domain state machine
(section 4.1 of http://www.eros-os.org/devel/intro/ProgrmrIntro.html).
In the new system, a domain (very similar to a UNIX process) can now be
available *and* running.

Precisely a domain can now have three states, scheduler-wise:

   1. running and busy ("activation flag" set);
   2. running and available ("activation flag" cleared);
   3. blocked (waiting for the kernel to awaken it).

When in state 1, the domain's control flow (at the assembler level) is
never altered by the kernel: this is similar, although not identical, to
blocking signals in UNIX. Messages arriving during state 1 are not
delayed (unlike UNIX - the Coyotos kernel never delays, delays are
EVIL!), but instead silently posted to the domain's FCRBs provided they
are short (this is only my best guess, see below) and discarded otherwise.

States 2 and 3 are similar in every respect, except that the domain
isn't executing instructions in 3 (transition from 2 to 3 is similar to
a processor executing the HLT instruction, but with interrupts enabled).
While in state 2 and 3, the domain is able to receive messages in a way
very similar to UNIX signals: when a message arrives, the kernel causes
a jump to it's "activation entry point" with the stack pointer set to
the "activation stack", which are both arranged prior (how: fixed
address? Kernel call? Part of the shared-memory mailbox detailed below? 
Doesn't matter that much in the big picture anyway).

If I understand it correctly, the main idea to be lifted from Nemesis'
activations is that there are no such system calls as sigprocmask() and
sigaction(); instead the Coyotos API for scheduling and activation is a
shared-memory mailbox in a page shared with the kernel. The domain
therefore has full control over its state transitions in the diagram
above, although it is unclear to me how one transitions into state 3 (I
understand there is a bit to set at IPC send time that means "block me
right after that", but one should think of that mechanism as an artefact
providing atomicity, rather than of a UNIXoid blocking system call. I
therefore assume that there has to be some kind of user-mode HLT
instruction - maybe blockingly invoking a null cap?).

>An FCRB can be named by a capability. The effect of
>invoking this capability is to deliver [*] the FCRB to it's bound
>receiving process [by issuing an activation]. 
My understanding gets a bit murkier past that point.

FCRBs look a lot like pollable events to me. Each of them represents a
condition that the domain is waiting to become true, the authority of
causing this to happen being vested in a capability (of course). When
that cap is invoked by someone somewhere, the domain gets notified

    * if in state 1 and the message's payload is sufficiently short, by
      writing stuff into the FCRB mailbox (which is memory shared with
      the kernel, just like the activation mailbox that contains the
      activation bit - I don't understand whether these two kinds of
      mailboxes are related any further under Shap's proposed
    * if in state 2 or 3, by interrupting resp. resuming the domain
      (which is then expected to do what? Copy payload data or caps from
      some temp buffer? Wasn't the kernel itself supposed to do the
      copying in case of an EROS IPC, subject to resource availability
      provisions that guarantee promptness? Or is the activation simply
      a courtesy of the kernel saying "hello, new message", the
      payload-moving part of the IPC procedure being unchanged from
      before? I'm lost here).

The ability to have FCRBs resolved at any time, even in state 1, means
that the process is continuously executing a select() system call, so to
speak, even while it is actively executing instructions. "Blessing" a
range of memory into being an FCRB is up to the domain (probably by
invoking the kernel?), that may do so any number of times as memory
permits (the return value is a capability, which the domain can then
hand out at will - sort of like a busy executive giving his cellphone
number only to select people because he doesn't quite like being
interrupted). FCRBs can even be made re-usable for conditions that are
expected to occur several times. FCRBs can be stored into or removed
from a kernel queue, just like one inserts or removes file descriptors
in the select() lists (although it is unclear to me why there should be
*several* kernel queues for a given Coyotos domain, or what the
"broadcast" feature of awakening several domains at once is useful for).

As indicated above, I'm not sure how all this relates to the IPC
payload-moving mechanism; but I can try to guess (most unwise when
dealing with Shap, as I found out :-) but here we go). The Coyotos
equivalent of the UNIX write() system appears to have just been
separated in two different layers: one that causes the caller to stop
and the server to resume executing instructions (using the FCRB
mechanism), and another, *built on top of it* to actually move the bytes
across. In other words, IPC is now second-class to activation, unlike
EROS where the two were tightly integrated: the slice-it-all microkernel
approach is winning the day, again.

Thanks for your patience!

<< Tout n'y est pas parfait, mais on y honore certainement les jardiniers >>

			Dominique Quatravaux <dom at kilimandjaro.dyndns.org>

More information about the coyotos-dev mailing list