[coyotos-dev] IDL compatibility

[Charles Landau]

At 12:22 PM -0400 6/19/07, Jonathan S. Shapiro wrote:
>The current scheme is that the most-significant 8 bits encode the
>inheritance depth, with the "root" value being 1 (to guarantee
>non-collision with RC_OK). The least-significant 24 bits are the method
>number in order of appearance within the interface.
>
>This is not a good scheme. It is vulnerable to many innocuous edits that
>can cause method codes to be inadvertently altered.

Right, that is exactly the problem I encountered in changing the 
package name from "eros" to "capros". The exception codes 
inadvertently changed.

>Using inheritance depth was also a mistake. We have already seen cases
>where method names have needed to migrate and/or interfaces have needed
>to be injected into a hierarchy.

How does a hash of the full name solve that problem?

>CapIDL will support hand-assignment of method codes at some point; MarkM
>and I recognized the need for this (due to compatibility issues) from
>the beginning.

We are in agreement that the current scheme does not meet compatibility needs.

IDL is an interface *definition* language. If instead it were an 
interface *description* language, you could write IDL files and 
maintain them without altering the definition of method codes and 
exception codes, which you said you do not want to inadvertently do.

In either case, the codes have to be chosen somehow, and neither of 
us has a perfect scheme for that.

But assigning the codes outside the IDL does have the advantage of 
introducing a level of indirection, so you can change method names, 
order, or levels independently of changing codes.

>Unfortunately, I do
>not know of any implicit code point assignment strategy *other* than a
>pure hash on fully-qualified name (or reliably random generation) that
>does not share this issue [vulnerability to innocuous edits of the IDL file].

And hashing also has that issue, if you include renaming as an innocuous edit.

Assigning codes outside the IDL at least does not have that problem.

>The goal is to minimize hand assignment and to *strongly*
>discourage any code point model that would lend itself to collisions
>when methods need to be relocated and/or rearranged.

If relocating and/or rearranging leads to collisions, the model is 
broken. If it leads to unintended code point changes, that is to be 
discouraged. And I would add renaming to the list.

>Realistically, the only mechanisms I know about that satisfy all of
>these objectives are strongly random assignment and cryptographic
>hash-based assignment. The latter has the advantage of determinism, and
>eliminates the need to hand-assign code points to every method during
>development (which lowers cost of adoption of CapIDL, which is good).

Since development never ends, we definitely want to avoid hand-assignment.

>  > >The current exception code scheme will fail if more
>>  >than 2^16 exceptions come to exist system-wide, and I am of the opinion
>>  >that exception code values should be increased to 64 bits to avoid this.
>>
>>  That would reduce the probability of collision, but not to zero, and
>>  at a runtime cost.
>
>There is, in principle, no way to reduce the probability of collision to
>zero.

The current scheme for method codes (which you described above) meets 
the requirement of zero probability of collision in a single chain of 
inheritance.

>The collision probability

considering global collisions

>from cryptographic hashes is orders of
>magnitude lower than any known human-assigned scheme -- including the
>one you proposed below (discussion follows below).

That scheme wasn't advertised or intended to prevent global 
collisions - only collisions in a single chain of inheritance.

>The runtime cost is largely irrelevant.

At 12:43 PM -0400 6/19/07, Jonathan S. Shapiro wrote:
>This requires that RC_OK also be extended to 64 bits.
>In practice this is not a problem, but in rare cases it will cause reply
>spill-over in EROS.

More relevantly, in CapROS.

On 32-bit architectures it will always require testing two words for 
zero instead of one.

>  > The way to absolutely guarantee non-collision is to declare all the
>>  exceptions at one level. Unix's errno.h demostrates that that's
>>  feasible. cap.idl already declares exceptions that are only used by
>>  derived interfaces.
>
>Please explain how I can declare all exception codes that will be needed
>by developers who are designing an unanticipatable application 25 years
>from now.

You don't have to. You can add new exceptions 25 years from now.

The issue is, how does Joe Developer who wants a new exception, get 
the system developers to add the exception to the global list?

I think the answer is, if he can't, he assigns the exception code (or 
gets IDL to assign it) randomly or cryptographically, and prays (with 
high success) that it doesn't conflict.

That doesn't mean that the system developers must also assign 
randomly or cryptographically, and pray, when they could use a scheme 
that is 100% reliable. In other words, a combination of techniques 
may be our best hope here.

>errno.h is a wonderful illustration of how to do it wrong. As errors
>were added to errno.h by different vendors, there were collisions.
>Cleaning up errno code point discrepancies is one of the real annoyances
>that has to be dealt with when implementing binary support for different
>unix personalities in a single system.

Your proposal does not solve this problem. If you define exception 
codes cryptographically, linux.ENOMEM will have a different value 
from hpux.ENOMEM.

>The entire problem with errno.h is that the code points were selected in
>a political namespace (i.e. one run by humans) rather than a randomized
>one. Had they been randomly assigned, no reconciliation would have been
>needed at all.

Except to match up linux.ENOMEM with hpux.ENOMEM.