[coyotos-dev] IPC redesign issues, goals, and options

[Jonathan S. Shapiro]

Issues/Questions:

1. Is background message transfer important? How big a deal is it that
preempting a reply-wait causes the reply to be lost?

  My opinion:

  1A) Background *delivery* is extremely useful. The "cannot
  preempt waiting process without message loss" issue was a debugging
  nightmare.

  1B) Background *receipt* is not very important, and appears
  to be causing some significant issues.

2. Is the activation handler stuff still useful and relevant? Should it
be dropped?

  My opinion:

  The need for a clean application preemption mechanism was real,
  but any preemption mechanism that entails delivery of capability
  state isn't going to work for us.

3. Should the activation handling mechanism and the message delivery
mechanism be tied together, or should they be separated?

  My opinion:

  Message delivery necessarily entails capability delivery, so they
  cannot be successfully integrated as much as they are right now.
  It remains possible that notice of pending messages could be handled
  by ``soft interrupt''.


Goals:

  G1: It should be possible to preempt a process without breaking
      any outstanding receive.

  G2: A soft preemption mechanism is desirable.


Options for Messaging:


At this point I think that we have several possible options for the IPC
design:

1. Revert to the EROS design, accepting all of its warts. The issues are
   at least well understood.

   This does not satisfy either goal.

2. Make the return from activated mode be handled by a system call. This
   gives us a place to swap the capability registers. A variant design
   is possible in which the capability register switch is handled
   lazily.

   This is doable, but it is pretty complicated. It implies that there
   is a capability page per FCRB, which is going to be a problem on
   small memory systems.

3. Make the ``message receive from FCRB'' be handled by an explicit
   receive operation. This lets the application do it when it knows that
   capability/data registers will be clobbered and can deal with that
   without requiring any fancy activation context stuff. This implies
   that activation handling (if it remains at all) is orthogonal to FCRB
   receipt. There are three sub-options:

   3A) Process is not notified of message availability. Short message
       payload will be delivered at next explicit receive operation.

   3B) Process is notified of message availability by soft-interrupt.
       Short message payload will be delivered at next explicit
       receive operation.

   3C) Process is notified of message availability by soft interrupt
       as in (3B), but only if it is not currently receiving.

4. Move to a design where the payload currently stored in the FCRB is
   delivered immediately, but not to registers. In this design, the FCRB
   contains two capabilities:

   a) A capability page capability (capability arguments are delivered
      into this page)

   b) A data page capability (data words are delivered here instead of
      to registers).

   In typical usage, the data page capability could point to the UPCB,
   which effectively delivers the message into per-thread registers.

   This is doable, but it is space intensive. Every outstanding receive
   now ties down *two* pages, which is an even bigger problem on
   small memory systems.


My inclination is to move to some variant of (3).


Options for Activations:

I am inclined to keep the activation mechanism, but detach it entirely
from the message payload delivery. I am inclined to deliver a
soft-interrupt on message delivery.


Needless to say, this is all going to entail some re-writing of the
specification. Sigh.


-- 
Jonathan S. Shapiro, Ph.D.
Managing Director
The EROS Group, LLC
+1 443 927 1719 x5100