The Object Teams Blog

Adding team spirit to your objects.

How many concepts for modules do we need?

with 13 comments

The basic elements of programming are methods. If you have many methods you want to group them in classes. If you have many classes you want to group them in packages. If you have many packages you want to group them in bundles. If you have many bundles you group them in features, but if you have many features …stop!, STOP!!!

Isn’t this insane? Every time we scale up to the next level we need a new programming concept? Like, someone invented the +1 operator and I can trump him by inventing a +2 operator, and you trump me by …? Haven’t we learned the 101 of programming: abstraction? I guess not many folks in Java-land are aware of a language called gbeta, where classes and methods are unified to “patterns” and no other kinds of modules are needed than patterns. It’s good news that the guy behind gbeta receives one of this year’s prestigious Dahl-Nygaard prizes: Erik Ernst. Object Teams owes much to Erik and I will speak more about his contributions in a sequel post.

Another really smart guy is Gilad Bracha, who after working on Java generics (together with Erik actually) and even JSR 294 decided to do something better, actually doubleplusgood. While he upholds the distinction between methods and classes he makes strong claims that no other modules than classes are needed, if, and here is the rub: if classes can be nested (see “Modules as Objects in Newspeak“).

Modules in Java: man or boy?

Let me briefly check this hypothesis:

The proliferation of module concepts in Java is due to the lack of truly nestable modules.

Which of the above mentioned concepts supports nesting? Features support inclusion which isn’t exactly nesting, but since features are actually defied by OSGi purists, I don’t want to burn my fingers by promoting features. Bundles cannot be nested, bummer! Some people actually think packages support nesting, with two reasons for believing so: packages are mapped to directories in a files system or in a jar and directories can be nested, plus: packages have compound names. However, semantically the dot in a package name is just a regular part of the name, it has no special semantics. Speaking of package foo.bar being a “sub-package” of foo is strongly misleading as the relation between them two is in no way different from the relation between any other pair of packages. All packages are equalst. Perhaps you recall the superpackage hero (or
the strawman of that hero), which introduced the capability that a public class can actually be seen by classes from specific superpackages only. And: superpackages where designed to support nesting. Now superpackages are “superceded” by Jigsaw modules, which don’t support nesting. Great, they invented the +3 operator. That’s award winning!

Finally classes: surprisingly, yes, classes can be nested. Unfortunately, still today it shines through that nested classes are an after-thought, the real core of Java 1.0 was designed without that. E.g., serializing nested classes is strongly discouraged. How many O/R mappers support non-static inner classes? The last time I looked: zero!

Nested classes: flaws and remedies (part 1)

I see three conceptual flaws in the design of nested classes in Java:

  1. scoping (2 problems actually)
  2. forced representation
  3. poor integration with inheritance

I discuss the first two in this post, the third, inheritance, deserves a post on its own right.
For each problem I will briefly show how it is resolved in OT/J. First off, I should mention that OT/J maintains compatibility with Java by applying any modified semantics only inside classes with the team modifier.

Scoping

Consider the following Java snippet:

public class City {
   TownHall theTownHall = new TownHall();
   class TownHall {
       class Candidate { }
       Candidate[] candidates;
       void voteForMayor(Candidate candidate) { ... }
   }
   class Citizen {
       void performCivilDuties() {
           ...
           theTownHall.voteForMayor(theTownHall.candidates[n]);
       }
   }
}
 

In this code we can actually make all methods, fields and nested classes private, to the end that external clients see none of these internal details of a City, whereas classes at the inside can blithely communicate with each other. Thus we have created a wonderful module City which can use all accomplishments of object-oriented programming for its internal structure – well hidden from the outside. In Java, however, this is flawed in two ways:

  1. Nested classes cannot protect any members from access by sibling classes, so I (a Citizen) can actually see the wallets of all mayor Candidates (well, maybe that’s not a bug but a feature). It would be much more useful if only inside-out visibility was given, i.e., a Citizen can see all members of his/her City, but not inverse – the City looking into its glass Citizens.
  2. Scoping rules in Java are purely static, i.e., permissions are given to classes not objects. As a result I could not only vote in my home city, but in any City (and every person can see the wallet of any other person etc.).

OT/J solves (1) by reinterpreting the access modifiers. Within a team class a private field Candidate.wallet, e.g., would not be visible outside its class, whereas a Citizen could still access theTownHall if this field were private.

(2) is basically solved by applying different rules to self-references (incl. outer self) than to explicitly qualified expressions, so City.this.theTownHall (OK, uses the enclosing this instance) applies different rules than newYork.theTownHall (not OK if theTownHall is private).

Well, issue (1) is a matter of tedious details, where I see no excuse for Java’s “solution”. Issue (2) has always been a differentiation between good old Eiffel, e.g., and its “successor” Java. This issue stands for a conceptual crossroad: are we interested in code nesting, or are we more interested in expressing how run-time objects are grouped? I personally don’t see much use in making the definition of a Citizen a nested part of the definition of a City, but speaking of many Citizens (instances) forming a City (instance) makes a lot of sense.

Forced Representation

By this I mean the fact that programmers are forced to store all inner classes textually embedded in their enclosing class. After we’ve already seen the discrepancy between the semantics and the representation of a package (flat structure stored in nested directories), now we see the opposite: semantic nesting is forcefully tied to representational nesting. This sounds logical until you start to write significant amounts of code as one big outer class with lots of (deeply) nested classes contained. You probably wouldn’t even think of this, because pretty soon the file becomes unmanageably huge. This is a very mundane reason why class nesting in Java simply doesn’t scale.

The solution in OT/J is pretty trivial. The following structure

// file community/City.java
package community;
public team class City {
    protected class Citizen {}
}
 

is strictly equivalent to these two files:

// file community/City.java
package community;
/**
 * @role Citizen
 */
public team class City {
}
 
// file community/City/Citizen.java
team package community.City;
protected class Citizen {}
 

The trick is: City is both a class and a special package. So semantically the team contains the nested Citizen but physical storage may happen in a separate file stored in a directory community/City/ that corresponds to the enclosing team class.

As a special treat the package explorer of the OTDT provides two display options for team packages: logical vs. physical. In the physical view the 2-files solution looks like this:

Just by selecting the logical view, where the team-package is not shown, the display changes to

Small files in separate directories are good for team support etc. Logical nesting is good for modularity. Why not have the cake and eat it?

These are just little tricks introduced by OT/J and the OTDT. But with these there’s no excuse any longer for not using class nesting even for large structures. And remember: this is real nesting, so you can use the same concept for 1-level, 2-level, … n-level containment. Good news for developers, bad news for technology designers, because now we simply don’t need any “superpackages”, “modules” etc. I actually wonder, how nestable classes could be unified with bundles, but that’s still a different story to be told on a different day.

Before that I will discuss how nesting and inheritance produce tremendous synergy (instead of awkwardly standing side-by-side). From a research perspective this has been solved some 9 years ago. I strongly believe that the time is ripe to bring these fruits from ivory towers to the hands of real world developers. Stay tuned for part 2.

Advertisements

Written by Stephan Herrmann

March 20, 2010 at 22:50

13 Responses

Subscribe to comments with RSS.

  1. Excellent post, Stephan. I, too, have been turned off by all the Java Modules hype and bickering. Even as an Eclipse user, and developer, and big-time advocate, I don’t really care very much about the purities of OSGi or bundles; it just doesn’t seem as important, in practical, everyday terms, as the OSGi (or Jigsaw) gurus preach it to be.
    I also had forgotten about my distaste for nested classes in Java. I agree with your analysis of the fundamental flaws; I think I’d forgotten about them because I have simply adapted to basically avoid using nested classes (other than very simple anonymous classes). But it’s good to be reminded of the problems, even if all of this talk does re-kindle my fears that Java has nearly jumped the shark.

    Eric Rizzo

    March 21, 2010 at 02:47

  2. Very interesting article. Will you cover assembly in part 2? I’m very interested in how your classes can express requirements on other classes that includes versions.

    Thomas Hallgren

    March 21, 2010 at 08:19

  3. @Eric: I take two things from the recent fuzz about Java Modules: I understand that the need for action is real and recognized by many, and for the time being I take OSGi and Equinox as actually quite good choices until we have better ones (and current JSRs are not likely to improve here).

    However, in object-oriented programming classes are at the core of our business. In OT/J we have mended some of the flaws of nested classes in Java. So with OT/J we should really give nested classes a chance to live up to the idea that classes are the primary modules and not only at one level (a few hundreds of lines of code) but potentially at any level of scale.

    Doing all this based on Java is a limitation, but maybe we can still bend Java to a direction that has a future. I hope so, not because I love Java, but for very pragmatic reasons.

    stephan

    March 21, 2010 at 12:24

  4. @Thomas: Thanks for reminding me of assembly. Let’s see: the next post will dive into inheritance issues. Actually, I can add a paragraph on how inheritance can be used for assembly – I hope you’ll be positively surprised 🙂

    As for version requirements etc. this is not currently covered by OT/J (which still is a ‘traditional’ programming language agnostic to packaging and deployment). With OT/Equinox we still leave versioning to the OSGi level, but as I mentioned, I’d love to discuss how both worlds can be unified, and I think they should. Let’s mark it on our agenda.

    stephan

    March 21, 2010 at 12:42

  5. Have you looked at Scala? It supports both nested and non-nested packages. Scala 2.8 introduces package objects, which are both objects (singletons) and packages. Combined with the Cake pattern, and some Scala features Java lacks (such as traits, abstract type members and type aliases), and the Cake pattern, you can achieve similar modularization to OT/J in Scala (at least I think they are similar, I have not looked deeply into OT/J).

    Erkki Lindpere

    March 22, 2010 at 01:14

  6. @Erkki: In order to unfold the story of Scala and OT/J I would probably start in the year 2000, when Martin Odersky and myself met at a conference explaining to each other the respective predecessors of Scala and OT/J 🙂

    However, the concepts mentioned in this post are the smaller part of what makes OT/J. The unique contribution of OT/J is its support for roles. Feel free to browse our documentation and write a comparison, if you see significant similarity between both languages.

    Just one comment: Scala is an admirable piece of green field development. OT/J deliberately extends Java in order to support smoothest possible adoption. Those are very different goals to begin with.

    As for package objects, no, I wasn’t aware of this late addition to Scala. But reading that these are singletons I’m actually disappointed: If you use these to model the above example you will end up with a world with only one City. Which reminds me that I thought of writing about the woes of singletons (which, e.g., OSGi people are very well aware of).

    BTW, when searching for a succinct description of the Cake pattern all I find are references to the paper “Scalable Component Abstractions” mentioning that the paper doesn’t use the term Cake pattern. I would have to guess what within this 18pp. paper is the Cake pattern.

    stephan

    March 22, 2010 at 09:30

  7. Awesome post, and extra kudos for mentioning the limitation of textually nesting classes! I am a big believer of (truly) nested classes since a long time, it is so nice to see them picked up!

    Adrian

    March 22, 2010 at 09:38

  8. Hi Stephan,

    Excellent coverage of nested classes. Of course, the list of quirks and misfeatures is much longer (try serializing an anonymous class; inheriting from a private class that you can access; …).

    I must quibble with your use of the term “Scoping”. You are really not discussing scope at all, but rather access control and class based encapsulation. The latter is an issue in Java irrespective of nested classes (do you change that?). Furthermore, access control in Java is dynamic; fixing the compiler is easy, but the runtime remains broken (true in Scala as well). Because of this, you cannot use objects as reliable capabilities for security, which is one of our goals with Newspeak (thanks for the plug BTW).

    Gilad Bracha

    March 22, 2010 at 15:59

  9. Hi Gilad,
    thanks for commenting!

    Who would know the quirks in Java better than you :). So what we I’re doing in OT/J is a daring balance between playing by the rules of Java and still making conceptually sound improvements.

    Why the heck did I write “Scoping”? Sure you’re right it’s about access control. I guess, however, I’m trying to better align both concepts, so that what I can see I’m also allowed to see. So private things are resolved only inside-out. With a strict inside-out scoping rule everything in scope is also accessible. Other access modifiers imply lookup strategies that may traverse also in other directions (outside-in, or sub-super). I’d hope that alignment of scoping and access control would provide for simpler intuition, but for precise argumentation the distinction is of course still needed (and of course a compiler first resolves and then checks access – but we wouldn’t necessarily want to tell programmers how the compiler works, would we?).

    Inside a team we indeed apply instance based access control (by the compiler). Regarding a team from the outside it looks like a regular class, so we maintain class based access control (but even here family polymorphism helps).

    Unfortunately I don’t see how access control for nested classes in Java could ever be made absolutely secure. You only have to know the name of one of those synthetic access methods in order to by-pass the compiler checks (even for class based access control). However, as I speak I realize that we could actually add a (more) secure mode pretty easily: we just need to extend those synthetic accessors: pass the caller as an additional argument and perform identity check within the accessor. This would be quite similar to how we already extend instanceof and cast (for family polymorphism). It’s still not 100% secure but I really wouldn’t want to do stack inspection for each invocation of a synthetic accessor, mh?

    Anyway, my primary goal is to improve modularity for reuse, evolution and maintenance. Regarding security I’m somewhat less ambitious to improve Java – I’ll leave that field for Newspeak 🙂

    stephan

    March 22, 2010 at 18:07

  10. Hi Stephan,

    Thanks for clarifying.

    BTW, I love your opening paragraph. I expect to quote it often in the future!

    Gilad Bracha

    March 25, 2010 at 00:04

  11. […] runtime, all you see is a soup of individuals (called objects) running around all over the place (remember: standard module systems do not create boundaries around runtime […]

  12. […] Warm-up: Debugging the proliferation of module concepts – will there ever be an end? Much of this follows the ideas in this post of mine. […]

  13. […] a previous post I mentioned that the strongest encapsulation in Java – using the private modifier – […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: