[coyotos-dev] Activations

[Jonathan S. Shapiro]

On Tue, 2007-05-22 at 15:46 -0400, Christopher Nelson wrote:
> 
> 
> On 5/22/07, Jonathan S. Shapiro <shap at eros-os.com> wrote:
>         On Tue, 2007-05-22 at 14:12 -0400, Christopher Nelson wrote:
>         
>         >
>         >         Umm. Chris? The memory overhead of the activation
>         handler is
>         >         actually
>         >         *higher* than the memory overhead of the external
>         handler... 
>         >
>         > Seriously? Then I have a misunderstanding about activation
>         handlers.
>         > My impression of their function was like this:
>         >
>         > MMAP - 1 per process (1 total)
>         > Stack   - 1 per thread   (N total for N threads) 
>         
>         Not sure what you mean by MMAP here. Can you expand?
> 
> I meant memory map, indicating the address space constructs as a
> whole.  

Any memory map is constructed once at process startup, and doesn't
impact the performance or feasibility of either approach.

>         Recall when comparing that there are two independent issues
>         here:
>         
>           1. The cost of the U->S->U transition
>           2. The cost of the address space switch
>         
>         When people quote cswitch times, they generally lump these
>         together. Of 
>         the two, the address space switch is the dominant cost. In
>         EROS and
>         Coyotos, we don't incur that cost if the invoker and invokee
>         share the
>         same address space.
> 
> I was assuming that, with an external fault handler, address space
> switching was required, but with an activation handler it was not.  So
> I was thinking just in terms of U->S->U for activation handlers, and
> in terms of complete context switch for external handlers.

Hmm. Actually, an activation handler does

	U->S->U[activated]->U[normal]

where an external handler must do

	U->S->U[exhndlr]->S->U

Neither the call nor the return in *either* design entail an address
space switch. On RISC machines the extra U->S->U is no big deal, and
probably not on *later* IA-32 and AMD64 systems where SYSCALL/SYSENTER
is present. Finally, recall that we're talking about exception handling
here, which means that we are already into a low-frequency event.

> If you have to have additional storage for each activation handler
> per-thread, then it seems like the only additional overhead for the
> extern handlers it the address space mapping.

I'm very confused as to why you keep coming back to this. In the
external case, the handler is using *identically* the same address space
as the faulting process. This is also true in the activation case. The
mmap's required to set up those spaces all happen at process startup
time, and are completely orthogonal to the choice of exception handling
mechanism.

That is: the external handler requires *zero* marginal mmap's. The only
marginal cost is the extra U->S->U transition.

> Of course, external handlers have to handle their own faults for their
> own memory requests anyways.  So moving that complexity really only
> seems to move it. 

Generally this is not true. In most cases a handler (of either variety)
runs from fixed space and arranges (statically or at initialization
time) to ensure that all required space is available prior to any
attempt to handle a fault.

> Don't you still have to have  additional memory
> overhead per external handler for guaranteed space so that the fault
> handler can handle it's own memory faults? 

No, because the fault handler, by design, does not take memory faults.

> w/o spending too much time on me, can you outline why activation
> handlers require more memory overhead than external handlers?

Well, I can try! :-)

The respective stack sizes are the same. The difference is that in the
external case we allocate an an additional process structure, where in
the activation case we have to allocate an additional page (for the save
area). Processes are significantly smaller than pages.

Further, in an application having multiple kernel-scheduled worker
threads (each of which, in Coyotos, is a Process), the activation design
requires a distinct stack and transfer page for each worker thread,
while the external design can get away with a single handler process.

The reason for this is that activation does not occur by IPC, and
therefore lacks any mutual exclusion protections. If the activation
region were shared, you would get scrambled state. While the kernel
*could* check "am I already activated", it has no efficient way to
notice when the activation handler has returned so that some other fault
can be delivered. This can be handled by the activation handler return
path, but it makes the return path significantly more complicated, and
that path is intrinsically delicate to begin with.

shap