The Object Teams Blog

Adding team spirit to your objects.

Interfacing null-safe code with legacy code

with one comment

When you adopt null annotations like these, your ultimate hope is that the compiler will tell you about every possible NullPointerException (NPE) in your program (except for tricks like reflection or bytecode weaving etc.). Hallelujah.

Unfortunately, most of us use libraries which don’t have the blessing of annotation based null analysis, simply because those are not annotated appropriately (neither in source nor using external annotations). Let’s for now call such code: “legacy”.

In this post I will walk through the options to warn you about the risks incurred by legacy code. The general theme will be:

Can we assert that no NPE will happen in null-checked code?

I.e., if your code consistently uses null annotations, and has passed analysis without warnings, can we be sure that NPEs can only ever be thrown in the legacy part of the code? (NPEs inside legacy code are still to be expected, there’s nothing we can change about that).

Using existing Eclipse versions, one category of problems would still go undetected whereby null-checked code could still throw NPE. This has been recently fixed bug.

Simple data flows

Let’s start with simple data flows, e.g., when your program obtains a value from legacy code, like this:


You shouldn’t be surprised, the javadoc even says: “The method returns null if the property is not found.” While the compiler doesn’t read javadoc, it can recognize that a value with unspecified nullness flows into a variable with a non-null type. Hence the warning:

Null type safety (type annotations): The expression of type ‘String’ needs unchecked conversion to conform to ‘@NonNull String’

As we can see, the compiler warned us, so we are urged to fix the problem in our code. Conversely, if we pass any value into a legacy API, all bad that can happen would happen inside legacy code, so nothing to be done for our mentioned goal.

The underlying rule is: legacy values can be safely assigned to nullable variables, but not to non-null variables (example Properties.getProperty()). On the other hand, any value can be assigned to a legacy variable (or method argument).

Put differently: values flowing from null-checked to legacy pose no problems, whereas values flowing the opposite direction must be assumed to be nullable, to avoid problems in null-checked code.

Enter generics

Here be dragons.

As a minimum requirement we now need null annotations with target TYPE_USE (“type annotations”), but we have this since 2014. Good.


Here we obtain a List<String> value from a Legacy class, where indeed the list names is non-null (as can be seen by successful output from names.size()). Still things are going south in our code, because the list contained an unexpected null element.

To protect us from this problem, I marked the entire class as @NonNullByDefault, which causes the type of the variable names to become List<@NonNull String>. Now the compiler can again warn us about an unsafe assignment:

Null type safety (type annotations): The expression of type ‘List<String>’ needs unchecked conversion to conform to ‘List<@NonNull String>’

This captures the situation, where a null value is passed from legacy to null-checked code, which is wrapped in a non-null container value (the list).

Here’s a tricky question:

Is it safe to pass a null-checked value of a parameterized type into legacy code?

In the case of simple values, we saw no problem, but the following example tells us otherwise once generics are involved:

Again we have a list of type List<@NonNull String>, so dereferencing values obtained from that list should never throw NPE. Unfortunately, the legacy method printNames() succeeded to break our contract by inserting null into the list, resulting in yet another NPE thrown in null-checked code.

To describe this situation it helps to draw boundaries not only between null-checked and legacy code, but also to draw a boundary around the null-checked value of parameterized type List<@NonNull String>. That boundary is breached when we pass this value into legacy code, because that code will only see List<String> and happily invoke add(null).

This is were I recently invented a new diagnostic message:

Unsafe null type conversion (type annotations): The value of type ‘List<@NonNull String>’ is made accessible using the less-annotated type ‘List<String>’

By passing names into legacy code, we enable a hidden data flow in the opposite direction. In the general case, this introduces the risk of NPE in otherwise null-checked code. Always?


Java would be a much simpler language without wildcards, but a closer look reveals that wildcards actually don’t only help for type safety but also for null-safety. How so?

If the legacy method were written using a wildcard, it would not be (easily) possible to sneak in a null value, here are two attempts:

The first attempt is an outright Java type error. The second triggers a warning from Eclipse, despite the lack of null annotations:

Null type mismatch (type annotations): ‘null’ is not compatible to the free type variable ‘?’

Of course, compiling the legacy class without null-checking would still bypass our detection, but chances are already better.

If we add an upper bound to the wildcard, like in List<? extends CharSequence>, not much is changed. A lower bound, however, is an invitation for the legacy code to insert null at whim: List<? super String> will cause names.add() to accept any String, including the null value. That’s why Eclipse will also complain against lower bounded wildcards:

Unsafe null type conversion (type annotations): The value of type ‘List<@NonNull String>’ is made accessible using the less-annotated type ‘List<? super String>’

Comparing to raw types

It has been suggested to treat legacy (not null-annotated) types like raw types. Both are types with a part of the contract ignored, thereby causing risks for parts of the program that still rely on the contract.

Interestingly, raw types are more permissive in the parameterized-to-raw conversion. We are generally not protected against legacy code inserting an Integer into a List<String> when passed as a raw List.

More interestingly, using a raw type as a type argument produces an outright Java type error, so my final attempt at hacking the type system failed:



We have seen several kinds of data flow with different risks:

  • Simple values flowing checked-to-legacy don’t cause any specific headache
  • Simple values flowing legacy-to-checked should be treated as nullable to avoid bad surprises. This is checked.
  • Values of parameterized type flowing legacy-to-checked must be handled with care at the receiving side. This is checked.
  • Values of parameterized type flowing checked-to-legacy add more risks, depending on:
    • nullness of the type argument (@Nullable type argument has no risk)
    • presence of wildcards, unbounded or lower-bounded.

Eclipse can detect all mentioned situations that would cause NPE to be thrown from null-checked code – the capstone to be released with Eclipse 2020-03, i.e., coming soon …

Written by Stephan Herrmann

February 6, 2020 at 20:38

Posted in Eclipse

Tagged with , , , ,

Oracle made me a Stackoverflow Guru

leave a comment »

Just today Oracle helped me to become a “Guru” on Stackoverflow! How did they do it? By doing nothing.

In former times, I was periodically enraged, when Oracle didn’t pay attention to the feedback I was giving them during my work on ecj (the Eclipse Compiler for Java) – at least not the attention that I had hoped for (to be fair: there was a lot of good communication, too). At those times I had still hoped I could help make Java a language that is completely and unambiguously defined by specifications. Meanwhile I recognized that Java is at least three languages: the language defined by JLS etc., the language implemented by javac, and the language implemented by ecj (and no chance to make ecj to conform to both others). I realized that we were not done with Java 8 even 3 years after its release. Three more years later it’s still much the same.

So let’s move on, haven’t things improved in subsequent versions of Java? One of the key new rules in Java 9 is, that

“If [a qualified package name] does not name a package that is uniquely visible to the current module (§7.4.3), then a compile-time error occurs”.

Simple and unambiguous. That’s what compilers have to check.

Except: javac doesn’t check for uniqueness if one of the modules involved is the “unnamed module”.

In 2018 there was some confusion about this, and during discussion on stackoverflow I raised this issue to the jigsaw-dev mailing list. A bug was raised against javac, confirmed to be a bug by spec lead Alex Buckley. I summarized the situation in my answer on stackoverflow.

This bug could have been easily fixed in javac version 12, but wasn’t. Meanwhile upvotes on my answer on stackoverflow started coming in. The same for Java 13. The same for Java 14. And yet no visible activity on the javac bug. You need ecj to find if your program violates this rule of JLS.

Today the 40th upvote earned me the “Guru” tag on stackoverflow.

So, please Oracle, keep that bug unresolved, it will earn me a lot of reputation for a bright future – by doing: nothing 🙂

Written by Stephan Herrmann

January 16, 2020 at 19:40

Posted in Eclipse

Tagged with , , ,

Jigsaw arriving – and now what?

leave a comment »

The debate is over. The result of fierce discussions has been declared. Jigsaw is knocking at the door. Now the fate of Jigsaw depends on the community. Will the community embrace Jigsaw? Will we be able to leverage the new concepts to our benefit? Will we struggle with new pitfalls?

Let’s step back a second: do we really know what is Jigsaw? Do we really understand the design, its subtleties and implications?

eclipsecon2017At EclipseCon Europe 2017 I will try to shed a light on some lesser known aspects of Jigsaw. A deep dive into things I learned as a JDT committer while implementing tool support for Java 9.

In search for truth

To set the stage, we’ll first have to figure out, who or what defines Java 9 — or Jigsaw — or JPMS. This is both a question of specification vs. implementation as well as a matter of a specification spread over several documents and artifacts. Let’s try to grok Jigsaw from the legally binding sources, rather then from secondary writing and hearsay (if that is possible).

We will have to relearn some of the basic terms, like: what is a package in Java? Do packages form a hierarchy? (I will show, how both answers, yes and no, are equally right and wrong).

Jigsaw is said to do away with some “undesirable” stuff like split packages, and cyclic dependencies. Really? (Yes and no).


Of course, with Jigsaw all is about encapsulation – easy to agree on, but what is it that a Java 9 module encapsulates? Only a deep understanding of the flavor of encapsulation will tell us, what exact qualities we gain from following the new discipline (it’s not about security, e.g.), and also what pains will be incurred on the path of migrating to the new model. (Hint: I will be speaking both about compile time and runtime).

Interestingly, the magic potion Jigsaw also brings its own antidote: With tightening the rules of encapsulation, also the opposite becomes necessary: they call it breaking encapsulation, for which I once coined the term “decapsulation“. I should be flattered by how close Java 9 comes to what I call “gradual encapsulation“. So, the talk can not stop at presenting just the new language concepts, also the various extra-lingual means for tuning your architecture need to be inspected through the looking glass. This is also where tools come into focus: how can JDT empower its users to use the new options without the need to dig through the sometimes cryptic syntax of command line parameters?

Loose ends

At this point we shall agree that Jigsaw is a compromise, fulfilling many of its goals and promises, while also missing some opportunities.

I will also have to say a few words about long standing API of JDT, which makes guarantees that are no longer valid in Java 9. This raises the question: what is the migration path for tools that sit on top of JDT? (There is no free lunch).

Finally, it wouldn’t be Java, if overloading wouldn’t make things worse for the newly added concepts. But: haven’t you become fond of hating, when JDT says:

error_obj The type cannot be resolved. It is indirectly referenced from required .class files

We may be seeing light at the end of the tunnel: for Jigsaw we had to revamp the guts of our compiler in ways, that could possibly help to – finally – resolve that problem. Wish me luck …

Hope to see you in Ludwigsburg, there’s much to be discussed over coffee and beer

eclipsecon2017 Ludwigsburg, Germany · October 24 – 26, 2017

Written by Stephan Herrmann

September 10, 2017 at 16:31

Several Languages Java™ 8

with 5 comments

More than 3 years ago, on March 18, 2014, Java™ 8 was released, and on the same day Eclipse released support for this new version. I have repeatedly made the point that the Eclipse compiler for Java (ecj) as a second implementation of JLS (after javac) serves the entire Java community as a premier means for quality assurance.

Today, in April 2017, I can report that this effort is still far from complete. Still, JLS, javac and ecj do not define the exact same language. Time to take stock what these differences are about.

My own work on ecj focuses on an aspect that tries hard to remain invisible to casual users: type inference. It’s the heavy machinery behind generic methods, diamond expressions and lambdas, allowing users to omit explicit type information in many places, leaving it to the compiler to figure out the fine print.

To be honest, when we first shipped support for Java 8, I was well expecting lots of bug reports to come in, which would point out corner cases of JLS that we hadn’t implemented correctly. There was one area, that I felt particularly uneasy about: how type inference blends with overload resolution. During the Mars cycle of development Srikanth thankfully performed a major rework and clean up of this exact area.
(I can’t pass the opportunity to report what I learned from this exercise: Overloading is a huge contributor of complexity in the Java language which in (not only) my opinion doesn’t carry its own weight — not a fraction of it).

We are not done with Java 8

The fact that still 2 years after that rework we constantly receive bug reports against Java 8 type inference is unsettling in a way.

To give some numbers to it: during every 6-week milestone we fixed between 1 and 6 bugs in type inference. None of these bugs is solved in a coffee break, some compete for the title “toughest challenge I faced in my career”.

We have a total of 103 bugs explicitly marked as 1.8 inference bugs. Of these

  • 17 were resolved before Java 8 GA
  • 52 have been resolved in the three years since Java 8 GA
  • 34 are still unresolved today.

This will likely keep me busy for at least one more year.

In the early days of Java 8 we could identify two clusters where behavioral differences between javac and ecj could be observed:

  • wildcard capture
  • raw types

(I’ll have a note about the latter at the end of this post).

In these areas we could comfort ourselves by pointing to known bugs in javac. We even implemented code to conditionally mimic some of these javac bugs, but frankly, establishing bug compatibility is even more difficult than truthfully implementing a specification.

Meanwhile, in the area of wildcard capture, javac has been significantly improved. Even though some of these fixes appear only in Java 9 early access builds, not in Java 8, we can observe both compilers converging, and given that the major bugs have been fixed, it is getting easier to focus on remaining corner cases. Good.

Java “8.1”

One event almost went under our radar: In February 2015 a revised version of JLS 8 was published. As part of this update, a few sentences have been added on behalf of JDK-8038747. While the spec may be hard to grok by outsiders, the goal can be explained as enabling a compiler to include more type hints from the bodies of lambda expressions that are nested inside a generic method invocation.

In fact, collecting type information from different levels in the AST was a big goal of the type inference rewrite in Java 8, but without the 2015 spec update, compilers weren’t even allowed to look into the body of a lambda, if the lambda does not specify types for its arguments.

m(a -> a.b())
What do we know about b, while types for m and a are still unknown?

Conceptually, this is immensely tricky, because generally speaking, the code in the bodies of such type-elided lambdas can mean just about anything, while the signature of the lambda is not yet known. So we were happy about the spec update, as it promised to resolve a bunch of test cases, where javac accepts programs that ecj – without the update – was not able to resolve.

Ever since, each improved version of ecj created a regression for one or more of our dear users. We debugged no end, read between the lines of JLS, but couldn’t find a solution that would satisfy users in all cases. And they kept complaining that javac had no problems with their programs, even earlier versions of ecj accepted their program, so rejecting it now must be a regression.

“Switching the alliance”

Up-to that point, I saw our main ally in the authors of JLS, Dan Smith and Alex Buckley from Oracle. In particular Dan Smith has been a tremendous help in understanding JLS 8 and analyzing where our reading of it deviated from the authors’ intention. Together we identified not only bugs in my interpretation and implementation of JLS, but also several bugs in javac.

When we iterated bugs relating to JDK-8038747 time and again, this approach was less effective, coming to no conclusion in several cases. I slowly realized, that we were reaching a level of detail that’s actually easier to figure out when working with an implementation, than at the specification level.

This is when I started to seek advice from javac developers. Again, I received very valuable help, now mostly from Maurizio Cimadamore. Previously, my perception was, that JLS is the gold standard, and any deviation or even just liberal interpretation of it is bad. During the discussion with Maurizio I learned, that in some sense javac is actually “better” than JLS, not only in accepting more type-correct programs, but also in terms of better reflecting the intention of the JSR 335 experts.

So I started to deliberately deviate from JLS, too. Instead of “blaming” javac for deviating from JLS, I now “blame” JLS for being incomplete wrt the intended semantics.

To put this effort into proportion, please consider the figure of 103 bugs mentioned above. From these, 17 bugs have a reference to JDK-8038747. Coincidentally, this is the exact same number as those great bug reports prior to Java 8 GA, that gave us the huge boost, enabling us to indeed deliver a high quality implementation right on GA. In other words, this is a huge engineering effort, and we have no idea, how close to done we are. Will we face the next round of regressions on every new release we publish?

If you work from a specification, there is a point where you feel confident that you did all that is required. Knowing that fulfilling the spec is not enough, it’s impossible to say, what is “enough”.

What is “better”?

With wildcard captures and raw types, it was easy to argue, that certain programs must be rejected by a compiler, because they are not type safe and can blow up at runtime in unexpected locations. In the area around JDK-8038747 javac tends to accept more programs than JLS, but here it would be unreasonable to expect javac to change and start rejecting these “good” programs.

Still, calling out a competition of who accepts more “good” programs would be a bad idea, too, because this would completely abandon the goal of equivalence between compilers. After compiling with one compiler, one could never be sure that another compiler would also accept the same program. The term “Java” would loose its precise meaning.

This implies, every attempt to better align ecj with javac, based on knowledge about the implementation and not based on JLS, should be seen as a temporary workaround. To resume its role of leadership, JLS must catch up with any improvements done in the implementation(s).

To comfort the reader, I should say that in all cases discussed here, there’s always a safe fallback: when inference fails to find a solution, it is always possibly to help the compiler by adding some explicit type arguments (or argument types for a lambda). More importantly, such additions, which may be required for one compiler, should never cause problems for another compiler.

Also note, that explicit type arguments are always to be preferred over type casts (which some people tend to use as a workaround): type arguments will help for effective type checking, whereas type casts bypass type checking and can blow up at runtime.

Thanks and Sorry!

I wrote this post in the desire to reach out to our users.

First: Each reproducible bug report is highly valuable; this is what drives JDT code towards higher and higher quality. By accumulating test cases from all these reports we gradually create a test suite that provides the best available safety net.

Second: I am sorry about every regression introduced by any of our fixes, but as this post should explain, we are traveling uncharted territory: some of the corner cases we are currently addressing are not sufficiently covered by JLS. Additionally, type inference is inherently sensitive to the slightest of changes. Predicting, which programs will be affected by a given change in the implementation of type inference is near impossible.

Yet, it’s certainly not a game of “them” vs “us”: JLS, javac, and ecj, we’re all in this together, and only by continuing to talk to each other, eventually we will all speak the same language, when we say “Java 8”. Please bear with us as the saga continues …

PS: Another pet peeve

I am a type system enthusiast, mostly, because I like how type checkers can completely eliminate entire categories of bugs from your programs. I like to give a guarantee that no code that is accepted by the compiler will ever fail at runtime with an exception like attempting to invoke a method that is not present on the receiver, or class cast exceptions in source code that doesn’t mention any class cast. Type inference is the tool that alleviates the verbosity of explicitly typed programs, while at the same time maintaining the same guarantees about type safety.

Unfortunately, there is a class of Java programs for which such guarantees can not be given: if a program uses raw types, the compiler needs to generate lots of checkcast instructions, to make the code acceptable for the JVM. Each of these instructions can cause the program to blow up at runtime in totally unsuspicious locations.

There are situations where javac silently assumes that a raw type List is a subtype of its parameterized form List<String>. This is wrong. Still I cannot just ignore this problem, because lots of “bugs” are reported against ecj, based on the observation that javac and ecj accept different programs, where in many cases the difference concerns the handling of raw types during type inference.

Economically speaking, investigating in the subtleties of how Java 8 compilers handle raw types is a huge waste of efforts. Any one reading this: if you want to do me a favor, and thus help me to focus on the most relevant aspects of compiler development, please clean up your code. If you keep your code private, nobody will suffer except from yourself, but please, before posting a bug report against JDT, if your code example contains raw types, think thrice before submitting the report. Adding proper type arguments will certainly improve the quality of your code. Likely, after that exercise also ecj will be a lot happier with your code and give you correct answers.

Do I need to repeat that raw types were a workaround for migrating towards Java 5? … that raw types were discouraged starting from day 1 of Java 5? If that doesn’t convince you, search on StackOverflow for questions mentioning raw types and type inference, and you will see that by the use of raw types you are basically disabling much of the power of type inference. Let’s please shed the legacy of raw types.

Written by Stephan Herrmann

April 2, 2017 at 21:43

Posted in Eclipse, Uncategorized

Tagged with , ,

Eclipse Neon.2 is on Maven Central

with 6 comments

It’s done, finally!

Bidding farewell to my pet peeve

In my job at GK Software I have the pleasure of developing technology based on Eclipse. But those colleagues consuming my technology work on software that has no direct connection to Eclipse nor OSGi. Their build technology of choice is Maven (without tycho that is). So whenever their build touches my technology we are facing a “challenge”. It doesn’t make a big difference if they are just invoking a code generator built using Xtext etc or whether some Eclipse technology should actually be included in their application runtime.

Among many troubles, I recall one situation that really opened my eyes: one particular build had been running successfully for some time, until one day it was fubar. One Eclipse artifact could no longer be resolved. Followed long nights of searching why that artifact may have disappeared, but we reassured ourselves, nothing had disappeared. Quite to the contrary somewhere on the wide internet (Maven Central to be precise) a new artifact had appeared. So what? Well, that artifact was the same that we also had on our internal servers. Well, if it’s the same, what’s the buzz? It turned out it had a one-char difference in its version: instead of 1.2.3.v20140815 its version was 1.2.3-v20140815. Yes take a close look, there is a difference. Bottom line, with both almost-identical versions available, Maven couldn’t figure out what to do, maybe each was considered as worse than the other, to the effect that Maven simply failed to use either. Go figure.

More stories like this and I realized that relying on Eclipse artifacts in Maven builds was always at the mercy of some volunteers, who typically don’t have a long-term relationship to Eclipse, who filled in a major gap by uploading individual Eclipse artifacts to Maven Central (thanks to you volunteers, please don’t take it personally: I’m happy that your work is no longer needed). Anybody who has ever studied the differences between Maven and OSGi (wrt dependencies and building that is) will immediately see that there are many possible ways to represent Eclipse artifacts (OSGi bundles) in a Maven pom. The resulting “diversity” was one of my pet peeves in my job.

At this point I decided to be the next volunteer who would screw up other people’s builds who would collaborate with the powers that be at to produce the official uploads to Maven Central.

As of today, I can report that this dream has become reality, all relevant artifacts of Neon.2 that are produced by the Eclipse Project, are now “officially” available from Maven Central.

Bridging between universes

I should like to report some details of how our artifacts are mapped into the Maven world:

The main tool in this endeavour is the CBI aggregator, a model based tool for transforming p2 repositories in various ways. One of its capabilities is to create a Maven repository (a dual use repo actually, but the p2 side of this is immaterial to this story). That tool does a great job of extracting meta data from the p2 repo in order to create “meaningful” pom files, the key feature being: it copies all dependency information, which is originally authored in MANIFEST.MF, into corresponding declarations in the pom file.

Still a few things had to be settled, either by improving the tool, by fine tuning the input to the tool, or by some steps of post-processing the resulting Maven repo.

  • Group IDs
    While OSGi artifacts only have a single qualified Bundle-SymbolicName, Maven requires a two-part name: groupId x artifactId. It was easy to agree on using the full symbolic name for the artifactId, but what should the groups be? We settled on these three groups for the Eclipse Project:

    • org.eclipse.platform
    • org.eclipse.jdt
    • org.eclipse.pde
  • Version numbers
    In Maven land, release versions have three segments, in OSGi we maintain a forth segment (qualifier) also for releases. To play by Maven rules, we decided to use three-part versions for our uploads to Maven Central. This emphasizes the strategy to only publish releases, for which the first three parts of the version are required to be unique.
  • 3rd party dependencies
    All non-Eclipse artifacts that we depend on should be referenced by their proper coordinates in Maven land. By default, the CBI aggregator assigns all artifacts to the synthetic group p2.osgi.bundle, but if s.o. depends on p2.osgi.bundle:org.junit this doesn’t make much sense. In particular, it must be avoided that projects consuming Eclipse artifacts will get the same 3rd party library under two different names (perhaps in different versions?). We identified 16 such libraries, and their proper coordinates.
  • Source artifacts
    Eclipse plug-ins have their source code in corresponding .source plug-ins. Maven has a similar convention, just using a “classifier” instead of appending to the artifact name. In Maven we conform to their convention, so that tools like m2e can correctly pick up the source code from any dependencies.
  • Other meta data
    Followed a hunt for project url, scm coordinates, artifact descriptions and related data. Much of this could be retrieved from our MANIFEST.MF files, some information is currently mapped using a static, manually maintained mapping. Other information like licences and organization are fully static during this process. In the end all was approved by the validation on OSSRH servers.

If you want to browse the resulting wealth, you may start at

Everything with fully qualified artifact names in these groups (and date of 2017-01-07 or newer) should be from the new, “official” upload.

This is just the beginning

The bug on which all this has been booked is Bug 484004: Start publishing Eclipse platform artifacts to Maven central. See the word “Start”?

To follow-up tasks are already on the board:

(1) Migrate all the various scripts, tools, and models to the proper git repo of our releng project. At the end of the day, this process of transformation and upload should become a routine operation to be invoked by our favourite build meisters.

(2) Fix any quirks in the generated pom files. E.g., we already know that the process did not handle fragments in an optimal way. As a result, consuming SWT from the new upload is not straight forward.

Both issues should be handled in or off bug 510072, in the hope, that when we publish Neon.3 the new, “official” Maven coordinates of Eclipse artifacts will be even fit all all real world use. So: please test and report in the bug any problems you might find.

(3) I was careful to say “Eclipse Project”. We don’t yet have the magic wand to apply this to literally all artifacts produced in the Eclipse community. Perhaps s.o. will volunteer to apply the approach to everything from the Simultaneous Release? If we can publish 300+ artifacts, we can also publish 7000+, can’t we? 🙂

happy building!

Written by Stephan Herrmann

January 9, 2017 at 23:21

Posted in Eclipse, Uncategorized

Tagged with , , , ,

Runtime Specialization – At Last

with one comment

Between a rock and a hard place

Not long ago, I had to pull Object Teams out of the Eclipse simultaneous release train. Reason: the long standing issue of using BCEL for bytecode weaving, for which no Java 8 compatible version has yet been released. With the Eclipse platform moving to Java 8, this had escalated to a true blocker. During the last weeks I investigated two options in parallel:

  • Upgrade BCEL to a release candidate of the upcoming 6.0 release.
  • Finalize the alternative weaver for OT/J, which is based on ASM, and thus already capable of handling Java 8 byte codes

I soon found out that even BCEL 6.0 will not be a full solution, because it still has no real support for creating StackMapTable attributes for newly generated bytecode, which however is strictly mandatory for running on a JVM 8.

For that reason I then focussed on the OTDRE, the Object Teams Dynamic Runtime Environment. This has been announced long ago, and after all, I promised to show a sneak preview of this feature in my presentation at EclipseCon Europe:

Runtime Specialization
Java has never been so dynamic before

Success at last

Today I can report success in two regards:

  • The Object Teams Development Tooling, which itself is a complex OT/J application, can (mostly) run on the new runtime!
  • I created a first demo example that shows the new capability of runtime weaving in action – it works! 🙂

This is a major milestone! Running OTDT on top of OTDRE is a real stress test for that new component – once again I realize that dog-fooding an entire IDE on its own technology is quite an exciting exercise. While a few exceptions need to be ironed out before the Neon release, I’m confident now, that we’re finally and really on the home stretch of this effort.

But OTDRE is not just a replacement for the traditional runtime, it is also way cooler, as the second success story shows: it is indeed possible now, to throw new teams and roles into a running application. If the application has been generically prepared for this task, the new teams can be automatically activated and immediately start adapting the running application – no specific preplanning needed. With this we are able to achieve a level of dynamism that typically is only possible with dynamic languages. And all this without any compromise to static typing and analysability. This is probably too cool to be allowed.

And after all the hard work on Java 8, also OT/J can finally fully leverage the new version, not only in theory, but also in bytecode.

Less than one week to finalize the presentation. You can be sure this will be a fresh story. Join me on Wednesday, Nov 4,  in Ludwigsburg:

Runtime Specialization - Java has never been so dynamic before -- Session at EclipseCon Europe 2015

PS: The “traditional” Object Teams Runtime Environment isn’t dead, yet. I really want to keep it as an option, because both variants (OTRE / OTDRE) have quite different characteristics, and after all this component has matured over more than 10 years. But with one option already (mostly) working, I can probably wait until a proper release of BCEL 6.0, and still have it back in game before the Neon release.

Written by Stephan Herrmann

October 27, 2015 at 23:05

Object Teams now has Lambdas (and more)

leave a comment »

In a previous post I announced that the Luna version of Object Teams will incorporate support for Java 8.

Now that the big rush towards March 18 is over, I took stock how Object Teams fits into the Luna stream, which finally contains all the bits for Java 8 support. The result:

We have our first builds with full support for OT/J on Java 8!

This means, you can now use lambda expressions and more even in OT/J programs, here’s a witness:

Lambda In Parameter Mapping

While this program may not make a lot of sense, it demonstrates a funny piece of syntax. Here’s what you see:

  • The regular class PersonCollection offers a method restrict that takes an argument of a functional type (Predicate)
  • Role Club.Members is played by PersonCollection and makes the base method available via a callout binding (lines 19 f)
  • The role wants to pass just an int value, where the base method expects a Predicate.
  • This incompatibility is bridged by a parameter mapping that maps a lambda expression to the expected parameter (line 20).

This looks funny, because two different uses of the token -> meet on a single line: the first -> is part of the lambda expression, separating the parameter from the body, the second -> is part of the parameter mapping, mapping the expression on the left to the parameter on the right.

Were I to chose the syntax in a green field situation, I would certainly not use the same token for such different purposes. But since we’re basically merging two independent language extensions I had no choice. After some worries that the combined language might not be parseable I’m much relieved to see it actually working!

A very similar situation actually exists concerning type annotations. More on that in a future post.

Remaining steps towards Luna

Merging the entire BETA_JAVA8 code base from JDT into Object Teams is done, but a few regressions (< 30 out of 84,000+ tests) remain from that exercise. But compared to the original JDT efforts for Java 8 this truly is peanuts 🙂

We only have one hurdle still to jump: our bytecode weaver is based on BCEL, for which no update for Java 8 is in sight. This means: we cannot weave into class files that use the Java 8 bytecode version, bummer!

So the real interesting question will be: can we get the long announced alternate weaver using ASM up to speed in time for Luna? I think we can, but it won’t hurt if you wish me luck in this endeavour.

And yes: one day I should start writing meaningful examples using Java 8 features in OT/J code…

Written by Stephan Herrmann

March 30, 2014 at 23:44

Posted in Eclipse, Object Teams, OTDT, OTJLD

Tagged with ,