[coyotos-dev] IDL compatibility: the versioning nightmare

[Jonathan S. Shapiro]

On Mon, 2007-07-09 at 14:07 -0700, Charles Landau wrote:
> >At 6:43 PM -0400 6/19/07, Jonathan S. Shapiro wrote:
> >>if you go to add a method to an existing interface,
> >>adding it in the middle of the file should not cause other code points
> >>to break.
> >
> >This implies that adding a name for a new method does not break the
> >IDL "contract".
> >
> >I see no reason why adding a *new* name for an *existing* method
> >should break the contract.

Let me illustrate why it breaks the contract. This illustration
expresses the heart of the interface versioning problem for capability
systems. Ideally we will come up with a solution through discussion.

MarkM and I have come up with a solution that is functionally effective,
but I do not consider it especially graceful.

I think it is pretty clear that changes to interfaces must be *upwards*
compatible. Adding a name to an existing interface certainly does not
violate this, and I will not spend time here justifying the need for
upwards compatibility.


The problem is this:

  Because interfaces are types, interfaces must also be *downwards*
  compatible.

This is a tricky point, because it is so counter-intuitive that it slips
away from everyone (even me). Let me illustrate.

Initially, I have two interfaces A and B:

  interface A {
    void methodOne();
  };

  interface B {
    int doSomething(A cap);
  };

Later, I add a new, upwards-compatible method to A, giving:

  interface A {
    void methodTwo();
  };

methodTwo might in fact be simply a new name for methodOne. It does not
matter. The key point is that a new identifier has been introduced, and
this is the source of a problem.

The question is: what does the method declaration of B.doSomething()
mean? It means that doSomething() expects to receive some capability
that implements the A interface. Well, what is that? It is an interface
implementing all of the methods of the A interface at the expected code
points. Well, so what is that?

And here you see the problem: in the absence of versioning, the
signature of B.doSomething() is underspecified; we have no way to know
whether the implementer of doSomething expects to be able to call
methodTwo or not.

It gets worse.

Suppose we add a versioning construct, so that we now write:

  interface A {
    version 1 {
      void methodOne();
    };
    version 2 {
      void methodTwo();
    };
  };

we might then write B.doSomething(A:1 cap) and it would be clear which
version was intended. But now consider:

  interface C : extends A {
  }

does C inherit all versions of A or some particular version?

If it is intended that versions are backwards compatible, then it
follows that every A:2 interface must implement all of A:1. If that is
the case, then it follows that the version construct is not necessary,
because interface inheritance can already express everything that there
is to be expressed:

  interface A {
    void methodOne();
  };
  interface A_v2 extends A {
    void methodTwo();
  };

and once again the B.doSomething() signature is unambiguous.

The problem here is that inheritance will progressively lead to a more
and more fragmented interface tree. So far as I can tell, the only way
to clean up the mess is to declare that interfaces may be re-collapsed
at major release boundaries.

And please nobody propose multiple inheritance as a solution. That just
makes matters much much worse!


Anyway, when MarkM and I last talked this through, we arrived at two
conclusions:

  1. Everything that can be correctly expressed with versioning
     can equally well be expressed with inheritance, so explicit
     versioning mechanism is technically required.

  2. Versioning is a nightmare. Forcing it to be expressed through
     inheritance helps make the nightmare more explicit. I, at least,
     believe that software nightmares should not be kept in closets.

shap
-- 
Jonathan S. Shapiro, Ph.D.
Managing Director
The EROS Group, LLC