The Object Teams Blog

Adding team spirit to your objects.

Object Teams now has Lambdas (and more)

leave a comment »

In a previous post I announced that the Luna version of Object Teams will incorporate support for Java 8.

Now that the big rush towards March 18 is over, I took stock how Object Teams fits into the Luna stream, which finally contains all the bits for Java 8 support. The result:

We have our first builds with full support for OT/J on Java 8!

This means, you can now use lambda expressions and more even in OT/J programs, here’s a witness:

Lambda In Parameter Mapping

While this program may not make a lot of sense, it demonstrates a funny piece of syntax. Here’s what you see:

  • The regular class PersonCollection offers a method restrict that takes an argument of a functional type (Predicate)
  • Role Club.Members is played by PersonCollection and makes the base method available via a callout binding (lines 19 f)
  • The role wants to pass just an int value, where the base method expects a Predicate.
  • This incompatibility is bridged by a parameter mapping that maps a lambda expression to the expected parameter (line 20).

This looks funny, because two different uses of the token -> meet on a single line: the first -> is part of the lambda expression, separating the parameter from the body, the second -> is part of the parameter mapping, mapping the expression on the left to the parameter on the right.

Were I to chose the syntax in a green field situation, I would certainly not use the same token for such different purposes. But since we’re basically merging two independent language extensions I had no choice. After some worries that the combined language might not be parseable I’m much relieved to see it actually working!

A very similar situation actually exists concerning type annotations. More on that in a future post.

Remaining steps towards Luna

Merging the entire BETA_JAVA8 code base from JDT into Object Teams is done, but a few regressions (< 30 out of 84,000+ tests) remain from that exercise. But compared to the original JDT efforts for Java 8 this truly is peanuts :)

We only have one hurdle still to jump: our bytecode weaver is based on BCEL, for which no update for Java 8 is in sight. This means: we cannot weave into class files that use the Java 8 bytecode version, bummer!

So the real interesting question will be: can we get the long announced alternate weaver using ASM up to speed in time for Luna? I think we can, but it won’t hurt if you wish me luck in this endeavour.

And yes: one day I should start writing meaningful examples using Java 8 features in OT/J code…

Written by stephanherrmann

March 30, 2014 at 23:44

Posted in Eclipse, Object Teams, OTDT, OTJLD

Tagged with ,

How to kick the fly-shuttle

leave a comment »

This post was originally written on August 6th, 2013, but got lost when moving this blog. Luckily I could recover it from backup.

Software composition by “weaving”

Programming languages in the wider field of Aspect Oriented Software Development introduce some kinds of composition concepts, that cannot directly be mapped to pure Java. We got used to speaking of “weaving” as a mechanism for preparing classes in these languages for execution on the JVM, whereas a true aspect oriented virtual machine would natively perform this composition transparently and more efficiently. But with the dominance of the JVM it will be “weaving” for a long time to come.

fly-shuttle

Back in the early days, AOSD languages performed some static composition on source or class files. When OT/J entered the stage, first frameworks emerged that allowed us to hook into the class loading process, so we could do the weaving dynamically on-the-fly, without ever storing the woven classes on disk. Work is (still) in the pipe-line to support even runtime-(re)weaving.

One theme in all the development of Object Teams is: first solve the dynamic case, that’s where the challenges lie, and leave more static cases for later as an optimization. “Later” has come now.

How (not) to hook into class loading?

A little while into the development of OT/J, Java 5 was published introducing the JPLIS API and the concept of a -javaagent. Today many instrumentation tools use this facility and OT/J was among the first on this boat. So for standalone Java applications the problem is solved.

Later, while developing Eclipse plug-ins, we noticed that it wouldn’t be overly cool if you have to invoke Eclipse with a -javaagent. Not only is this very inconvenient for installation, it is also problematic (e.g., for performance) to feed all classes of Eclipse through our weaver. We were lucky again, and the Equinox Adaptor Hooks emerged right when we needed them. With these hooks we were able to integrate OT/J in a way that enables all the on-the-fly flexibility we want while at the same time it lets us control the process so that the flexibility cannot be abused. So in Equinox (or better: OT/Equinox) the problem is (more than) solved.

The fact that Eclipse Luna will no longer support these hooks was slightly alarming news for Object Teams, but that problem is already essentially solved – details to be reported soon.

I’ve heard about successful experiments running OT/J code in tomcat, but, hey, the list of application servers/containers is long, and if we need specifically crafted integration for each environment, it’s kind of difficult to argue that OT/J runs “everywhere”. Is there a generic why to kick the fly-shuttle?

fly-shuttle-2

Dynamic weaving is cool, but not having a fallback is uncool, so I finally developed the static scenario as a special case of the dynamic one: a build-time weaver for OT/J.

The little new tool is ridiculously simple: we have a weaver capable of load-time weaving. All I had to do is: create a little utility that would invoke this weaver ahead-of-time, i.e., during building.

While I’m not a big fan of Maven, in my day job Maven is a must, so I bit the bullet used the opportunity to learn a bit about development of Maven plug-ins. I was amazed to find lots of tutorials still suggesting to use API that are deprecated in Maven 3 and more surprised to read forum threads suggesting to still use that deprecated API, partly to be compatible with Maven 2 (OK?) and partly because deprecation doesn’t always mean there is an alternative (OOPS?). It’s also interesting to see, that much fundamental, up-to-date information is not available from a central source but only from tribal knowledge in hundreds of forums, blogs and whatnots. — Enough whining, on to the new stuff.

For simple projects, all you need now is a pom that looks about like this:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 
    <modelVersion>4.0.0</modelVersion>
 
    <parent>
        <groupId>org.eclipse.objectteams</groupId>
        <artifactId>objectteams-parent-pom</artifactId>
        <version>2.2.0</version>
    </parent>
 
    <artifactId>OTStopwatch_Built-Time_Weaver_Example</artifactId>
    <version>0.0.1-SNAPSHOT</version>
 
    <properties>
        <tycho.version>0.18.0</tycho.version>
        <otj.version>2.2.0</otj.version>
    </properties>
 
    <build>
        <plugins>
            <plugin>
                <groupId>org.eclipse.objectteams</groupId>
                <artifactId>objectteams-weaver-maven-plugin</artifactId>
                <version>0.8.0-SNAPSHOT</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>weave</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <teamClasses>
                        <teamClass>org.eclipse.objectteams.example.stopwatch.WatchUI</teamClass>
                        <teamClass>org.eclipse.objectteams.example.stopwatch.WatchUIAnalog</teamClass>
                    </teamClasses>
                </configuration>
            </plugin>
        </plugins>
    </build>
 
    <repositories>
        <repository>
            <id>ObjectTeamsRepository</id>
            <name>Object Teams Repository</name>
            <url>http://download.eclipse.org/objectteams/maven/3/repository</url>
        </repository>
    </repositories>
 
   <dependencies>
       <dependency>
           <groupId>org.eclipse.objectteams</groupId>
           <artifactId>objectteams-runtime</artifactId>
           <version>${otj.version}</version>
       </dependency>
   </dependencies>
</project>
 

Here we enable OT/J compilation via the parent pom, specify a repo where to get the OT artifacts, a dependency and a few versions, nothing sophisticated.

What’s new is the objectteams-weaver-maven-plugin, which is minimally configured just by specifying a list of team classes that should be woven. Oops, that’s already it.

By integrating this into your build, you’ll produce class files that can be directly executed “everywhere”, i.e., without the OT/J load-time weaver, and thus: without fiddling with the class loading process. One caveat remains: naturally, by producing the woven class files offline, it will be your responsibility to correctly feed the correct versions of class files into the classpath in the desired order. If you can’t use the load-time weaver, it won’t be able to help keep things simple.

More configuration options are documented on the plug-in’s site and more explanation using our favorite examples can be found in the wiki.

Here you have it: static weaving falls of naturally if you already have dynamic weaving. And the resulting class files can truly be run “everywhere”.

I’d be curious to hear of the first OT/J apps on Android (though personally I’d prefer ubuntu).

Written by stephanherrmann

January 28, 2014 at 18:16

Posted in Eclipse, Object Teams, OTEquinox

Tagged with , , ,

Object Teams in the times of Eclipse Luna and Java 8

with one comment

With Eclipse Luna approaching milestone 5 it’s time to write a bit about what this year will bring for Object Teams.

Overhaul of the OT/Equinox weaver

The Luna cycle started with a surprise (for me): Equinox no longer provides the Adaptor Hooks, which allowed OT/Equinox to hook into the OSGi class loading process and perform its bytecode weaving. Outch!

On the other hand, OSGi meanwhile has a standard WeavingHook for doing this kind of stuff. With some efforts (and lots of help from Tom W. – no help from p2, though) I managed to migrate OT/Equinox to this new standard. At the end of the day, this will bring better encapsulation and hopefully better launch performance, too. Details to be posted some time later.

Lift OT/J to be based on Java 8

Some may have noticed that most my Eclipse-time is currently being spent in helping JDT to get ready for Java 8. And sure this is an exciting endeavour to be part of!

For Object Teams the cool part about this is: I’ve seen the changes for Java 8 happen in JDT, which greatly helps to adopt these changes for OT/J. Once in a while this even reveals a bug in JDT before it ever surfaced :)

The integration of OT/J with Java 8 still has some regressions, but my plan is to have this at good “milestone quality” when Java 8 is released in March and to get it to full release quality for Luna GA in June.

Question: does it actually make sense to combine lambdas and type annotations with roles and teams? I strongly believe it does, because these are language improvements in entirely different categories:

  • Lambda expressions help to implement algorithms in a more concise and also more modular way – this lets you think differently about functions.
  • Type annotations help enhance safety when used together with better program analysis (like, e.g., enhanced null analysis) – these let you think differently about types.
  • Roles and teams help improve the program structure at a larger scale – these let you think differently about classes and objects.

So, if each of these additions makes sense, then combining them all in one language will certainly yield a very powerful language. Some examples of combination to follow as we approach the release.

ObjectTeams.org is down, long live ObjectTeams.org

Not all material from the 10+ years of history of Object Teams has moved to Eclipse.org. Notably the language definition (OTJLD) and scientific publications are still hosted on objectteams.org. Until recently, my former University kindly provided the host for publishing these research results. Now that server has gone off service and for a while objectteams.org was dead — but as of today the relevant content is back online – sorry for the disruption. And, btw, this is also the reason why this blog has changed its URL.

Please wish me luck for the work ahead, both on JDT and Object Teams :)

Written by stephanherrmann

January 21, 2014 at 22:37

Book chapter published: Confined Roles and Decapsulation in Object Teams — Contradiction or Synergy?

leave a comment »

I strongly believe that for perfect modularity, encapsulation in plain Java is both too weak and too strong. This is the fundamental assumption behind a book chapter that has just been published by Springer.

The book is:
Aliasing in Object-Oriented Programming. Types, Analysis and Verification
My chapter is:
Confined Roles and Decapsulation in Object Teams — Contradiction or Synergy?

The concepts in this chapter relate back to the academic part of my career, but all of my more pragmatic tasks since those days indicate that we are still very far away from optimal modularity, and both mistakes are found in real world software: to permit access too openly and to restrict access too strictly. More often than not, it’s the same software exhibiting both mistakes at once.

For the reader unfamiliar with the notions of alias control etc., let me borrow from the introduction of the book:

Aliasing occurs when two or more references to an object exist within the object graph of a running program. Although aliasing is essential in object-oriented programming as it allows programmers to implement designs involving sharing, it is problematic because its presence makes it difficult to reason about the object at the end of an alias—via an alias, an object’s state can change underfoot.

Aliasing, by the way, is one of the reasons, why analysis for @Nullable fields is a thorny issue. If alias control could be applied to @Nullable fields in Java, much better static analysis would be possible.


How is encapsulation in Java too weak?

Java only allows to protect code, not objects

This manifests at two levels:

Member access across instances

In a previous post I mentioned that the strongest encapsulation in Java – using the private modifier – doesn’t help to protect a person’s wallet against access from any other person. This is a legacy from the pre-object-oriented days of structured programming. In terms of encapsulation, a Java class is a module utterly unaware of the concept of instantiation. This defect is even more frustrating as better precedents (think of Eiffel) have been set before the inception of Java.

A type system that is aware of instances, not just classes, is a prerequisite for any kind of alias control.

Object access meets polymorphism

Assume you declare a class as private because you want to keep a particular thing absolutely local to the current module. Does such a declaration provide sufficient protection? No. That private class may just extend another – public – class or implement a public interface. By using polymorphism (an invisible type widening suffices) an instance of the private class can still be passed around in the entire program – by its public super type. As you can see, applying private at the class level, doesn’t make any objects private, only this particular class is. Since every class extends at least Object there is no way to confine an object to a given module; by widening all objects are always visible to all parts of the program. Put dynamic binding of method calls into the mix, and all kinds of “interesting” effects can be “achieved” on an object, whose class is “secured” by the private keyword.

The type system of OT/J supports instance-based protection.

Java’s deficiencies outlined above are overcome in OT/J by two major concepts:

Dependent types
Any role class is a property of the enclosing team instance. The type system allows reasoning about how a role is owned by this enclosing team instance. (read the spec: 1.2.2)
Confined roles
The possible leak by widening can be prevented by sub-classing a predefined role class Confined which does not extend Object. (read the spec: 7.2)

For details of the type system, why it is suitable for mending the given problems, and why it doesn’t hurt in day-to-day programming, I have to refer you to the book chapter.


How is encapsulation in Java too strict?

If you are a developer with a protective attitude towards “your” code, you will make a lot of things private. Good for you, you’ve created (relatively) well encapsulated software.
But when someone else is trying to make use of “your” code (re-use) in a slightly unanticipated setting (calling for unanticipated adaptations), guess what: s/he’ll curse you for your protective attitude.
Have you ever tried to re-use an existing class and failed, because some **** detail was private and there was simply no way to access or override that particular piece? When you’ve been in this situation before, you’ll know there are basically 2 answers:

  1. Give up, simply don’t use that (overly?) protective class and recode the thing (which more often than not causes a ripple effect: want to copy one method, end up copying 5 or more classes). Object-orientation is strong on re-use, heh?
  2. Use brute force and don’t tell anybody (tools that come in handy are: binary editing the class file, or calling Method.setAccessible(true)). I’m not quite sure why I keep thinking of Core Wars as I write this :) .

OT/J opens doors for negotiation, rather than arms race & battle

The Object Teams solution rests on two pillars:

  1. Give a name to the act of disregarding an encapsulation boundary: decapsulation. Provide the technical means to punch tiny little holes into the armor of a class / an object. Make this explicit so you can talk and reason about the extent of decapsulation. (read the spec: 2.1.2(c))
  2. Separate technical possibility from questions of what is / should / shouln’t be allowed. Don’t let technology dictate rules but make it possible to formulate and enforce such rules on top of the existing technology.

Knowing that this topic is controversial I leave at this for now (a previous discussion in this field can be found here).


Putting it all together

  1. If you want to protect your objects, do so using concepts that are stronger than Java’s “private”.
  2. Using decapsulation where needed fosters effective re-use without the need for “speculative API”, i.e., making things public “just in case”.
  3. Dependent types and confined roles are a pragmatic entry into the realm of programs that can be statically analysed, and strong guarantees be given by such analysis.
  4. Don’t let technology dictate the rules, we need to put the relevant stakeholders back into focus: Module providers, application developers and end users have different views. Technology should just empower each of them to get their job done, and facilitate transparency and analyzability where desired.

Some of this may be surprising, some may sound like a purely academic exercise, but I’m convinced that the above ingredients as supported in OT/J support the development of complex modular systems, with an unprecedented low effective coupling.

FYI, here’re the section headings of my chapter:

  1. Many Faces of Modularity
  2. Confined Roles
    1. Safe Polymorphism
    2. From Confined Types to Confined Roles
    3. Adding Basic Flexibility to Confined Roles
  3. Non-hierarchical Structures
    1. Role Playing
    2. Translation Polymorphism
  4. Separate Worlds, Yet Connected
    1. Layered Designs
    2. Improving Encapsulation by Means of Decapsulation
    3. Zero Reference Roles
    4. Connecting Architecture Levels
  5. Instances and Dynamism
  6. Practical Experience
    1. Initial Comparative Study
    2. Application in Tool Smithing
  7. Related Work
    1. Nesting, Virtual Classes, Dependent Types
    2. Multi-dimensional Separation of Concerns
    3. Modules for Alias Control
  8. Conclusion

Written by stephanherrmann

March 30, 2013 at 22:11

Compiling OT/Equinox projects using Tycho

with 3 comments

In a previous post I showed how the tycho-compiler-jdt Maven plug-in can be used for compiling OT/J code with Maven.

Recently, I was asked how the same can be done for OT/Equinox projects. Given that we were already using parts from Tycho, this shouldn’t be so difficult, right?

Once you know the solution, finding the solution is indeed easy, also in this case. Here it is:

We use almost the same declaration as for plain OT/J applications:

    <pluginManagement>
	<plugin>
	    <groupId>org.eclipse.tycho</groupId>
	    <artifactId>tycho-compiler-plugin</artifactId>
	    <version>${tycho.version}</version>
	    <extensions>true</extensions>
	    <dependencies>
		<dependency>
		    <groupId>org.eclipse.tycho</groupId>
		    <artifactId>tycho-compiler-jdt</artifactId>
		    <version>${tycho.version}</version>
		    <exclusions>
			<!-- Exclude the original JDT/Core to be replaced by the OT/J variant: -->
			<exclusion>
			    <groupId>org.eclipse.tycho</groupId>
			    <artifactId>org.eclipse.jdt.core</artifactId>
			</exclusion>
		    </exclusions>
		</dependency>
		<dependency>
		    <!-- plug the OT/J compiler into the tycho-compiler-jdt plug-in: -->
		    <groupId>org.eclipse</groupId>
		    <artifactId>objectteams-otj-compiler</artifactId>
		    <version>${otj.version}</version>
		</dependency>
	    </dependencies>
	</plugin>
    </pluginManagement>

So, what’s the difference? In both cases we need to adapt the tycho-compiler-jdt plug-in because that’s where we replace the normal JDT compiler with the OT/J variant. However, for plain OT/J applications tycho-compiler-jdt is pulled in as a dependency of maven-compiler-plugin and must be adapted on this path of dependencies, whereas in Tycho projects tycho-compiler-jdt is pulled in from tycho-compiler-plugin. Apparently, the exclusion mechanism is sensitive to how exactly a plug-in is pulled into the build. Interesting.

Once I figured this out, I created and published a new version of our Maven support for Object Teams: objectteams-parent-pom:2.1.1 — publishing Maven support for Object Teams 2.1.1 was overdue anyway :)

With the updated parent pom, a full OT/Equinox hello world pom now looks like this:

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
 
    <parent>
        <!-- We use Object Teams: -->
        <groupId>org.eclipse</groupId>
        <artifactId>objectteams-parent-pom</artifactId>
        <version>2.1.1</version>
    </parent>
 
    <artifactId>OTEquinox-over-tycho</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <!-- We are building an Eclipse plug-in (or OSGi bundle): -->
    <packaging>eclipse-plugin</packaging>
 
    <repositories>
 
        <!-- This is where we get all Object Teams stuff from: -->
        <repository>
            <id>ObjectTeamsRepository</id>
            <name>Object Teams Repository</name>
            <url>http://download.eclipse.org/objectteams/maven/3/repository</url>
        </repository>
 
        <!-- Add any p2 repositories needed for your application, e.g.: -->
        <repository>
            <id>Juno</id>
            <name>Eclipse Juno Repository</name>
            <url>http://download.eclipse.org/releases/juno</url>
            <layout>p2</layout>
        </repository>
    </repositories>
 
    <build>
        <plugins>
            <plugin>
                <!-- We build this project using Tycho: -->
                <groupId>org.eclipse.tycho</groupId>
                <artifactId>tycho-maven-plugin</artifactId>
                <extensions>true</extensions>
            </plugin>
        </plugins>
    </build>
</project>

Looks pretty straight forward, right?

To see the full OT/Equinox Hello World Example configured for Maven/Tycho simply import OTEquiTycho.zip as a project into your workspace.

cheers,
Stephan

Written by stephanherrmann

October 31, 2012 at 00:16

Modularity at JavaOne 2012

leave a comment »

Those planning to attend JavaOne 2012 in San Francisco might be busy right now building their personal session schedule. So here’s what I have to offer:

On Wed., Oct. 3, I’ll speak about

Redefining Modularity and Reuse in Variants—
All with Object Teams

I'm Speaking at JavaOne 2012

I guess one of the goals behind moving Jigsaw out of the schedule for Java 8 was: to have more time to improve our understanding of modularity, right? So, why not join me when I discuss how much an object-oriented programming language can help to improve modularity.

When speaking of modularity, I don’t see this as a goal in itself, but as a means for facilitating reuse, and when speaking of reuse I’m particularly interested in reusing a module in a context that was not anticipated by the module provider (that’s what creates new value).

To give you an idea of my agenda, here’re the five main steps of this session:

  1. Warm-up: Debugging the proliferation of module concepts – will there ever be an end? Much of this follows the ideas in this post of mine.
  2. Reuse needs adaptation of existing modules for each particular usage scenario. Inheritance, OTOH, promises to support specialization. How come inheritance is not considered the primary architecture level concept for adapting reusable modules?
  3. Any attempts to apply inheritance as the sole adaptation mechanism in reuse mash-ups results in conflicts which I characterize as the Dominance of the Instantiator. To mend this contention inheritance needs a brother, with similar capabilities but much more dynamic and thus flexible.
  4. Modularity is about strictly enforced boundaries – good. However, the rules imposed by this regime can only deliver their true power if we’re also able to express limited exceptions.
  5. Given that creating variants and adapting existing modules is crucial for unanticipated reuse, how can those adaptations themselves be developed as modules?

At each step I will outline a powerful solution as provided by Object Teams.

See you in San Francisco!

5 items in your shopping cart Edit: The slides and the recording of this presentation are now available online.

Written by stephanherrmann

September 20, 2012 at 19:55

Object Teams 2.1 Milestone 7 (finally) brings hot code replacement

with 2 comments

As part of the Juno Milestone 7 also Object Teams has delivered its Milestone 7.

As the main new feature in this milestone hot code replacement finally works when debugging OT/J or OT/Equinox applications.

What took us so long to make it work?

  • Well, hot code replacement didn’t work out of the box because our load-time weaver only worked the first time each class was loaded. When trying to redefine a class in the VM the weaver was not called and thus class signatures could differ between the first loaded class and the redefined version. The VM would then reject the redefinition due to that signature change.
  • Secondly, we didn’t address this issue earlier, because I suspected this would be pretty tricky to implement. When I finally started to work on it, reality proved me wrong: the fix was actually pretty simple :)

In fact the part that makes it work even in an Equinox setting is so generic that I proposed to migrate the implementation into either Equinox or PDE/Debug, let’s see if there is interest.

Now when you debug any Object Teams application, your code changes can be updated live in the running debug target – no matter if you are changing teams, roles or base classes. Together with our already great debugging support, this makes debugging Object Teams programs still faster.

More new features can be found in the New&Noteworthy (accumulated since the Indigo release).

Written by stephanherrmann

May 12, 2012 at 22:10

Posted in Eclipse, Object Teams, OTDT, OTEquinox

Tagged with , ,

Follow

Get every new post delivered to your Inbox.