How many concepts for modules do we need?

The basic elements of programming are methods. If you have many methods you want to group them in classes. If you have many classes you want to group them in packages. If you have many packages you want to group them in bundles. If you have many bundles you group them in features, but if you have many features …stop!, STOP!!!

Isn’t this insane? Every time we scale up to the next level we need a new programming concept? Like, someone invented the +1 operator and I can trump him by inventing a +2 operator, and you trump me by …? Haven’t we learned the 101 of programming: abstraction? I guess not many folks in Java-land are aware of a language called gbeta, where classes and methods are unified to “patterns” and no other kinds of modules are needed than patterns. It’s good news that the guy behind gbeta receives one of this year’s prestigious Dahl-Nygaard prizes: Erik Ernst. Object Teams owes much to Erik and I will speak more about his contributions in a sequel post.

Another really smart guy is Gilad Bracha, who after working on Java generics (together with Erik actually) and even JSR 294 decided to do something better, actually doubleplusgood. While he upholds the distinction between methods and classes he makes strong claims that no other modules than classes are needed, if, and here is the rub: if classes can be nested (see “Modules as Objects in Newspeak“).

Modules in Java: man or boy?

Let me briefly check this hypothesis:

The proliferation of module concepts in Java is due to the lack of truly nestable modules.

Which of the above mentioned concepts supports nesting? Features support inclusion which isn’t exactly nesting, but since features are actually defied by OSGi purists, I don’t want to burn my fingers by promoting features. Bundles cannot be nested, bummer! Some people actually think packages support nesting, with two reasons for believing so: packages are mapped to directories in a files system or in a jar and directories can be nested, plus: packages have compound names. However, semantically the dot in a package name is just a regular part of the name, it has no special semantics. Speaking of package foo.bar being a “sub-package” of foo is strongly misleading as the relation between them two is in no way different from the relation between any other pair of packages. All packages are equalst. Perhaps you recall the superpackage hero (or
the strawman of that hero), which introduced the capability that a public class can actually be seen by classes from specific superpackages only. And: superpackages where designed to support nesting. Now superpackages are “superceded” by Jigsaw modules, which don’t support nesting. Great, they invented the +3 operator. That’s award winning!

Finally classes: surprisingly, yes, classes can be nested. Unfortunately, still today it shines through that nested classes are an after-thought, the real core of Java 1.0 was designed without that. E.g., serializing nested classes is strongly discouraged. How many O/R mappers support non-static inner classes? The last time I looked: zero!

Nested classes: flaws and remedies (part 1)

I see three conceptual flaws in the design of nested classes in Java:

1. scoping (2 problems actually)
2. forced representation
3. poor integration with inheritance

I discuss the first two in this post, the third, inheritance, deserves a post on its own right.
For each problem I will briefly show how it is resolved in OT/J. First off, I should mention that OT/J maintains compatibility with Java by applying any modified semantics only inside classes with the team modifier.

Scoping

Consider the following Java snippet:

public class City {
   TownHall theTownHall = new TownHall();
   class TownHall {
       class Candidate { }
       Candidate[] candidates;
       void voteForMayor(Candidate candidate) { ... }
   }
   class Citizen {
       void performCivilDuties() {
           ...
           theTownHall.voteForMayor(theTownHall.candidates[n]);
       }
   }
}
 

In this code we can actually make all methods, fields and nested classes private, to the end that external clients see none of these internal details of a City, whereas classes at the inside can blithely communicate with each other. Thus we have created a wonderful module City which can use all accomplishments of object-oriented programming for its internal structure – well hidden from the outside. In Java, however, this is flawed in two ways:

1. Nested classes cannot protect any members from access by sibling classes, so I (a Citizen) can actually see the wallets of all mayor Candidates (well, maybe that’s not a bug but a feature). It would be much more useful if only inside-out visibility was given, i.e., a Citizen can see all members of his/her City, but not inverse – the City looking into its glass Citizens.
2. Scoping rules in Java are purely static, i.e., permissions are given to classes not objects. As a result I could not only vote in my home city, but in any City (and every person can see the wallet of any other person etc.).

OT/J solves (1) by reinterpreting the access modifiers. Within a team class a private field Candidate.wallet, e.g., would not be visible outside its class, whereas a Citizen could still access theTownHall if this field were private.

(2) is basically solved by applying different rules to self-references (incl. outer self) than to explicitly qualified expressions, so City.this.theTownHall (OK, uses the enclosing this instance) applies different rules than newYork.theTownHall (not OK if theTownHall is private).

Well, issue (1) is a matter of tedious details, where I see no excuse for Java’s “solution”. Issue (2) has always been a differentiation between good old Eiffel, e.g., and its “successor” Java. This issue stands for a conceptual crossroad: are we interested in code nesting, or are we more interested in expressing how run-time objects are grouped? I personally don’t see much use in making the definition of a Citizen a nested part of the definition of a City, but speaking of many Citizens (instances) forming a City (instance) makes a lot of sense.

Forced Representation

By this I mean the fact that programmers are forced to store all inner classes textually embedded in their enclosing class. After we’ve already seen the discrepancy between the semantics and the representation of a package (flat structure stored in nested directories), now we see the opposite: semantic nesting is forcefully tied to representational nesting. This sounds logical until you start to write significant amounts of code as one big outer class with lots of (deeply) nested classes contained. You probably wouldn’t even think of this, because pretty soon the file becomes unmanageably huge. This is a very mundane reason why class nesting in Java simply doesn’t scale.

The solution in OT/J is pretty trivial. The following structure

// file community/City.java
package community;
public team class City {
    protected class Citizen {}
}
 

is strictly equivalent to these two files:

// file community/City.java
package community;
/**
 * @role Citizen
 */
public team class City {
}
 

// file community/City/Citizen.java
team package community.City;
protected class Citizen {}
 

The trick is: City is both a class and a special package. So semantically the team contains the nested Citizen but physical storage may happen in a separate file stored in a directory community/City/ that corresponds to the enclosing team class.

As a special treat the package explorer of the OTDT provides two display options for team packages: logical vs. physical. In the physical view the 2-files solution looks like this:

Just by selecting the logical view, where the team-package is not shown, the display changes to

Small files in separate directories are good for team support etc. Logical nesting is good for modularity. Why not have the cake and eat it?

These are just little tricks introduced by OT/J and the OTDT. But with these there’s no excuse any longer for not using class nesting even for large structures. And remember: this is real nesting, so you can use the same concept for 1-level, 2-level, … n-level containment. Good news for developers, bad news for technology designers, because now we simply don’t need any “superpackages”, “modules” etc. I actually wonder, how nestable classes could be unified with bundles, but that’s still a different story to be told on a different day.

Before that I will discuss how nesting and inheritance produce tremendous synergy (instead of awkwardly standing side-by-side). From a research perspective this has been solved some 9 years ago. I strongly believe that the time is ripe to bring these fruits from ivory towers to the hands of real world developers. Stay tuned for part 2.