[coyotos-dev] IDL compatibility

[Jonathan S. Shapiro]

On Tue, 2007-06-19 at 14:30 -0700, Charles Landau wrote:
> At 12:22 PM -0400 6/19/07, Jonathan S. Shapiro wrote:
> >The current scheme is that the most-significant 8 bits encode the
> >inheritance depth, with the "root" value being 1 (to guarantee
> >non-collision with RC_OK). The least-significant 24 bits are the method
> >number in order of appearance within the interface.
> >
> >This is not a good scheme. It is vulnerable to many innocuous edits that
> >can cause method codes to be inadvertently altered.
> 
> Right, that is exactly the problem I encountered in changing the 
> package name from "eros" to "capros". The exception codes 
> inadvertently changed.

The case you are describing is a case in which it was *necessary* for
the exception codes to change. You are imagining that you relocated the
exception definition from one interface to another. You did not. You
*deleted* an existing exception definition (which is generally a bad
thing to do, but not in your unusual circumstances) and inserted a new,
unrelated exception that has a sufficiently similar name to be causing
you confusion.

This is a model issue. Your model is reasonable, but it does not
correspond to the model necessary for IDL to work correctly.

> >Using inheritance depth was also a mistake. We have already seen cases
> >where method names have needed to migrate and/or interfaces have needed
> >to be injected into a hierarchy.
> 
> How does a hash of the full name solve that problem?

When coupled with the ability to hand-specify the method code, it allows
relocation in extremis. It solves the problem of interface layer
injection directly.

> >CapIDL will support hand-assignment of method codes at some point; MarkM
> >and I recognized the need for this (due to compatibility issues) from
> >the beginning.
> 
> We are in agreement that the current scheme does not meet compatibility needs.

No, we are not. You are pushing a particular objective that you believe
is important. I am trying to explain why your objective is not
appropriate for IDL.

> IDL is an interface *definition* language. If instead it were an 
> interface *description* language, you could write IDL files and 
> maintain them without altering the definition of method codes and 
> exception codes, which you said you do not want to inadvertently do.

Perhaps, but this would defeat the purpose of IDL. The purpose of IDL is
to specify an interface *contract* that is to be maintained over the
long haul. For this reason, and also for reasons of cross-language
compatibility, the goal is to make the IDL the *definitive* statement of
the interface. It is, indeed, an interface *definition* language.

Once an interface is in production, the kinds of migrations you are
talking about become categorically inappropriate. At that point it
becomes necessary to introduce new interfaces (i.e. we have to solve the
versioning problem).

> In either case, the codes have to be chosen somehow, and neither of 
> us has a perfect scheme for that.
> 
> But assigning the codes outside the IDL does have the advantage of 
> introducing a level of indirection, so you can change method names, 
> order, or levels independently of changing codes.

Yes. And all of these activities are frought with peril of interface
contract violation. Doing *any* of these after an interface is in
production use is almost always a fatal mistake. For this reason, CapIDL
bends over backwards to make this an exceptional pain in the ass to do.

> >Unfortunately, I do
> >not know of any implicit code point assignment strategy *other* than a
> >pure hash on fully-qualified name (or reliably random generation) that
> >does not share this issue [vulnerability to innocuous edits of the IDL file].
> 
> And hashing also has that issue, if you include renaming as an innocuous edit.

But renaming is very much NOT an innocuous edit. The name defines the
*identity* of the type or the method. The cases where a method can be
successfully migrated are very very rare.

> The current scheme for method codes (which you described above) meets 
> the requirement of zero probability of collision in a single chain of 
> inheritance.

Yes, but it does not satisfy the objective that the textual definition
of a method name should be able to be moved up or down within the
interface definition file without altering its code point. Another way
to look at this: if you go to add a method to an existing interface,
adding it in the middle of the file should not cause other code points
to break.

> On 32-bit architectures it [making RC_OK 64 bits] will always require testing 
> two words for zero instead of one.

I believe that with care this can be avoided. Even if it cannot, the
performance impact of this is noise relative to the importance of
interface maintenance and manageability.

> >Please explain how I can declare all exception codes that will be needed
> >by developers who are designing an unanticipatable application 25 years
> >from now.
> 
> You don't have to. You can add new exceptions 25 years from now.

This is obviously untrue. 25 years from now I may have no idea what you,
a developer in an unrelated company on an internal project, are doing.

> The issue is, how does Joe Developer who wants a new exception, get 
> the system developers to add the exception to the global list?

This requirement for central coordination is precisely what we must
avoid at all costs. The issue is how we make it *unnecessary* for Joe
Developer to have any need to contact the system developers.

Let us test your hypothesis. Since it is just about 25 years later,
please explain how I would go about getting an exception code added to
KeyKOS. I'm in a hurry, mind you, so a rapid resolution is essential.

> I think the answer is, if he can't, he assigns the exception code (or 
> gets IDL to assign it) randomly or cryptographically, and prays (with 
> high success) that it doesn't conflict.

Empirically this approach has not worked in the real world.

> >errno.h is a wonderful illustration of how to do it wrong. As errors
> >were added to errno.h by different vendors, there were collisions.
> >Cleaning up errno code point discrepancies is one of the real annoyances
> >that has to be dealt with when implementing binary support for different
> >unix personalities in a single system.
> 
> Your proposal does not solve this problem. If you define exception 
> codes cryptographically, linux.ENOMEM will have a different value 
> from hpux.ENOMEM.

Yes, and this is exactly the outcome that we require.

> >The entire problem with errno.h is that the code points were selected in
> >a political namespace (i.e. one run by humans) rather than a randomized
> >one. Had they been randomly assigned, no reconciliation would have been
> >needed at all.
> 
> Except to match up linux.ENOMEM with hpux.ENOMEM.

That is not an objective. It is an explicit objective of the design to
make this nearly impossible.

Rename is NOT an innocuous operation!!
-- 
Jonathan S. Shapiro, Ph.D.
Managing Director
The EROS Group, LLC