[bitc-dev] Retrospective Thoughts on BitC

David Jeske davidj at gmail.com
Mon Apr 9 03:32:03 PDT 2012


Forgive me for being under-educated in PL research for some of the audience
here...

http://www.coyotos.org/pipermail/bitc-dev/2012-March/003300.html

Shapiro wrote:

> From the perspective of a systems person, I regret to report that where
> the bits are placed, how big they are, and their assemblage actually *does*
> matter.


As a systems programmer, I couldn't agree more. A friend recently offered
me the colorful and true-ringing sound byte.

The PL community doesn't understand how ugly it is out here.  People are
> still writing shit in C.  To me, that's an emergency situation.  We can
> hold off unifying gravity and quantum mechanics for a while in order to
> stave off doomsday.


The reason I sought out BitC is that I'm looking for solutions not to
program-expression, but to software modularity. IMO, it is lack of
sufficient solutions to  systems-software-modularity, not PL design, that
is causing "shit to still be written in C" -- and a bulk of the pain in
practical software expression. My desire is for safer, better, and more
powerful solutions to software modularity and forward-compatibility; or in
practical terms, a replacement for the C-shared-library.

Some observations...

1) dynamic languages don't fix this.... current dynamic interpreted
languages are less forward-compatible than C-shared-libraries, because
imports have no version specification (e.g. two modules which require
different versions of ClassX can not run in the same Python runtime)
2) type-inference doesn't fix this... type inference systems are merely a
technique for software brevity
3) subtyping doesn't fix this.. it merely creates more challenges in both
modular forward compatibility and performance subtyping across module
boundaries
4) type-classes don't fix this... they are a PL or PL-implementation
feature and are not solving a moduarlity problem
5) JIT doesn't fix this... JIT is merely a tool which allows us to speed up
greater degrees of polymorphism

IMO, if we want to move past C-shared-libraries, the solution we need is a
model for type-safe high-performance loadable software components with
upgradability (managable forward compatibility).

My wish list for this modular software system does not involve syntax
discussions or inference models, but features required for run-time
assembling of compatible **and upgradable** software modules..

a) a strong/static/safe typing model for software components, which
provides....
b) a suitably polymorphic/parametric typing model, which (for performance)
provides...
c) control over memory layout of arrays of aggregate type, which requires..
d) parametric type instantiation, which requires...
e) JIT, which requires...
f) a unified garbage collection model, which requires...
g) a solution to soft-real-time garbage collection (to avoid
pause accumulation across subsystems)

Within this system, some interesting sub-problems are:

h) a model for module interface polymorphism, which provides...
i) module upgradability (i.e. a structure for managed forward interface
compatibility), which requires..
j) a solution to sub-module implementation dependency mismatch, which
requires...
k) operating system level management of multiple simultaneous installed
versions

We in the software community seem to be solving these problems in
isolation, which keeps the bottom of our stack deeply rooted in the
C-shared-library. The most interesting progress I've seen is MSIL, because
it solves the largest number of these sub-problems together. Microsoft
singularity is quite interesting research in this direction, as it is
attempting to eclipse the C-shared-library.

In practice, MSIL's decision to expose subtyping (class inheritence) across
module boundaries shifts quite a bit of responsibility for (i,j) to the
programmer, which feels to me like it falls apart in-the-large. My
intuition says that COM was heading down a better path, and that a system
based on aggregates, interfaces, and some form of implementation-conversion
or implementation-extension might provide a more managable model for
(i,j)... Whether that intuition is right or not, this is a huge area of
struggle in real software. In real systems, were punting on the whole
problem and isolating subsytems with (slow) message-passing. (between
kernel and user, between socket-connected processes, and at most C-shlib
boundaries) It would be nice to have more flexible (and higher performance)
solutions to this sub-problem.

Some related observations...

A1) basic-datatype message-passing is often used as a solution to (d,e) -
between kernel and user, or between socket-connected processes -- an
interface which only passes basic data-types in an agreed format does not
suffer from submodule version dependency differences

A2) JITs with Polymorphic Inline Caching improve the performance of
run-time polymorphism, but often do it without solving (e,f,g) (i.e. Java,
Javascript)

A3) software-virtual-machines provide some combination of features (JVM:
a,b,e,f,h), (MSIL: a,b,c,d,e,f,h,i,j), but are still both missing a
critical missing link to replace C-shared libraries... "e" (i.e.
deterministic soft-real-time performance), making them unsuitable for
layered subsystems. (because worst-case GC pauses are unacceptably large
both in large-heaps and layered small-heaps)

A4) operating systems are still providing unsuitable solutions to "k",
making reliably handling (i,j) more difficult than it could be...

============

Getting back to the bits... it's not surprising that BitC ran into trouble,
as trying to provide high-performance runtime polymorphism in modular
software without JIT seems impossible in the general case.

To me it seems that the importance of the assemblage of the bits is at the
root of why we're stuck with C out here in the real world. C is basically
an assembler that doesn't have general-case solutions to anything, but is
so unconstrained that it's possible to express anything with. Each higher
level language which only solves portions (or none) of the systems
modularity problem is thus relegated to serve as a whole-program compiler
within the world of  C-shlibs. I for one would like to see a better
solution to THAT problem.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.coyotos.org/pipermail/bitc-dev/attachments/20120409/a3feeb7b/attachment-0001.html 


More information about the bitc-dev mailing list