[coyotos-dev] FCRB concept is broken

[Jonathan S. Shapiro]

I am feeling very cranky this morning, because I have just realized (I
think) that FCRBs are broken. This means that the entire Coyotos message
system is a bad design that should be abandoned. More precisely:

  1. There is a bad (but simple) flaw in the current FCRB design.
     It has three possible fixes.

  2. Both fixes have unpleasant implications. They probably mean that
     FCRBs are not a good idea.

  3. Given these two statements, we need to ask whether FCRBs should be
     abandoned and we should return to a simpler mechanism.

  4. I still think that the activations stuff is useful, but I no longer
     think that it should be tied to messaging at all -- more on this in
     another note.

I would REALLY like to be wrong here, so I would very much like your
help to see if I am missing anything.


Two weeks ago, I added a "c0" slot to FCRBs to permit one capability to
be delivered in a short message (aside: we need a c1 as well). This is
necessary in order for the reply capability to be transmitted. The
assumption was that c0 would be delivered to a capability register when
the FCRB was delivered to the activation handler. Obviously, this
overwrites the destination register.

Unfortunately, the activation/de-activation mechanism does not provide
any way to move/save that register, which means that we are going to
clobber valid and important state.

There are two possible fixes. I haven't tried to figure out which is
better:

  Fix 1: Add fields to FCRB specifying *which* registers should be
         clobbered. In effect, these registers become "dedicated"
         to that FCRB.

         This solves the immediate problem, but the capability registers
         are no longer really acting like registers at all.

  Fix 2: Change c0 to be a capability **page**, and move the per-process
         capability registers back into a separate capability page.
         Delivery of an FCRB causes the capability register page to be
         replaced as a whole. It is up to the application to know how
         to reload that page (and a system call will be provided for
         this).

  Fix 3: [Variant of Fix 1]: go back to a capability register page so
         that there are more capability registers, then use fix 1.

  Fix 4: Switch to an "explicit receive" design.

If we adopt fix 2, we are saying that a user-mode thread switch requires
a kernel entry (needed to restore capability registers). At that point
the entire mechanism ceases to be cost-effective.

If we adopt fix 1,3, we severely limit the number of outstanding
simultaneous FCRBs that are possible.

It is tempting to imagine that fix 1,3 could simply deliver capabilities
into the capability address space. This doesn't work because the FCRB
payload needs to be reliably deliverable in the presence of a broken
address space.

I think that fix 2 is fatal to the FCRB idea for space reasons. Fix 3 is
feasible, but I think it is very complicated to use in a background
delivery scheme.

Fix 4 seems like the cleanest thing to do.

In the next few messages I will try to explore the options.


shap
-- 
Jonathan S. Shapiro, Ph.D.
Managing Director
The EROS Group, LLC
+1 443 927 1719 x5100