[coyotos-dev] Cross-model IPC

[Jonathan S. Shapiro]

We could use some help with an issue. Perhaps someone here has
experience or will see an obvious solution.

Some platforms, such as amd64, can run binaries in two or more modes.
Usually these are 64-bit architectures that provide backward support for
32-bit binaries. The question is: what, if anything, should the IDL and
IPC systems do to support cross-model calls.

Here are the underlying issues:

  1. The 64-bit and 32-bit environments typically disagree about
     alignment requirements for int64/uint64. On most 32-bit
     platforms only 32-bit alignment is required.

     In rare cases, this means that structures which fit in the
     direct string area on a 32-bit target will overflow into
     an indirect string on a 64-bit target. This actually isn't
     fatal, because the 64-bit receiver can re-align the message
     in this case. But if the message requires a *second* indirect
     string, the indirect strings will not agree positionally
     between 32-bit senders and 64-bit senders.

  2. All of the fast IPC transports define the "direct" payload
     in terms of register-sized words, not bytes. This means that
     we need to consider the problem of register truncation when
     messages are copied from 64-bit sender to 32-bit receiver.

  3. We do not want to require strict natural alignment on 32-bit
     machines, because this damages registerization at the IPC
     interface fairly severely.

We have come to two tentative decision:

  1. Receiver makes right. Most replies are one word, and are very
     easy to do any word-size adjustments on. Pragmatically, the
     tricky case is incoming requests (because these are likely to
     have longer payloads and indirect strings). Rather than force
     all clients to correct, it is better to do the correction
     in the receiver.

  2. We can get 99% of the registerization benefit if we operate
     according to the following constraints:

       a. Message is initially layed out under strict alignment
          rules. This determines (e.g.) whether indirect strings
          are used and at what "cut points" in the payload.

       b. Outermost objects in the direct words are aligned according 
          to the natural alignment of the sender (inner fields are
          still aligned strictly relative to start of their containing
          structure). This may mean that some direct words go unused
          in rare cases.

     If these two rules are adopted, the remaining problem appears to
     be register truncation when data moves from 64-bit sender to
     32-bit receiver.

Beyond this, the options appear to be:

  1. Implement distinct 32-bit and 64-bit entry points within the
     IDL-generated code. This seems like a lot of work, since in
     most cases the messages are actually the same.

  2. Have the message payload encode whether the sender is 32-bit
     of 64-bit. For calls from 32-bit to 64-bit, have receiver
     re-align if needed and then proceed as if it had received
     a 64-bit message.

     Have the sender indicate whether a 32-bit reply or a 64-bit
     reply is desired (this is not always simple).

  3. Push the entire problem to user land, and require translation
     agents when crossing the 32-bit/64-bit implementation boundary.

We would like to know:

  1. What have other IPC/IDL systems done about these situations?

  2. Do people see a way out of all this that would make the right
     decision "obvious"?