53 comments

[ 2.9 ms ] story [ 104 ms ] thread
Modularizing the JDK has been in progress for quite a long time -- getting "no" votes so close to the JDK 9 release date is pretty discouraging.
So for someone who has tinkered on the periphery of Java development (but does not work with Java/JVM languages professionally), it isn't entirely clear to me what the purpose of modules is for developers, over the encapsulation already provided by packages. Is it to simplify deployment?
If I'm not mistaken, Jigsaw sought to modularize the JDK itself, similar to what we see in the .Net Core space. There are many benefits to doing so.
Java packages are a fine "module" system if you're designing a single large cohesive application that has no dependencies or trust boundaries. However, packages are inadequate by themselves to tackle the problems that arise when you assemble an application from many different components, each of which are independently produced and versioned, and have different trust boundaries.

Java has an incredible open source community and there are amazing high-quality packages available to solve any problem under the sun; and so it is common for Java apps to have large dependency graphs. For example, take Google Guava, a popular and powerful Java library. It's so useful that other libraries depend on it. Let's say that our application uses two libraries called Component-A and Component-B, which both depend on Guava. However, Guava is constantly changing, and so it may arise that Component-A depends on Guava version 17 while Component-B depends on Guava version 20.

This is called a version conflict, and Java packages aren't enough to solve the problem well, though module systems can. For example, a module system allows you, as the application owner, to override the Guava version used by Component-A, replacing Guava-17 with Guava-20. If Component-A is compatible with Guava-20, then that resolves the problem. Another solution module systems provide is to allow both Guava-17 and Guava-20 to coexist in the same application, used separately by Component-A and Component-B. If those components are using Guava 'privately', as part of their implementation only, then this might also be an adequate resolution.

The default Java package system does not have a notion that there may be multiple implementations of a class like `com.google.common.base.Preconditions`, and that you may wish to select one version of it or another, or that you may want to use different versions in different parts of the application. It's not normally desirable to tamper with dependency versions or load multiple versions simultaneously, but version conflicts happen and module systems give you tools to deal with them when they occur. Fortunately, the behavior of Java packages is determined by Java ClassLoaders, and module systems can implement their own ClassLoaders to provide dynamic behaviors like these that go beyond vanilla Java's capabilities.

Since Java supports reflection, with regular Java packages, any class can access any other class and class member in the app, regardless of whether it's package private. These reflection capabilities are super useful; they're what enable frameworks like Spring, Guice, EasyMock, and Jackson. However, unrestricted reflective access is also a liability (viz. Ruby Monkey Patching), and it's something that you'd prefer to outright disable or at least sandbox in most components you depend on. But since these components might legitimately require reflection in their implementation, like using Guice for dependency injection or EasyMock for testing, you may not be able to disable reflection entirely; instead, module systems can help you sandbox reflection so that it operates within the confines of the module, to make the system simpler and more predictable.

Some module systems like OSGi also extend into the runtime behavior of your application, and help you manage complex applications. Certain very complex Java applications act a bit like an operating system: there are multiple services within the app that communicate with each other, and the module system isolates them for encapsulation reasons, loads them in the appropriate order, connects them together through exported interfaces, and provides management actions like reloading a module while the rest of the app keeps running. These module systems are for Java apps what systemd is for Linux (including the fact that some people love them while others think they're overly complex and should be avoided).

Thanks for the in-depth comment. Regarding version conflicts, we currently have a large enterprise system with lots of dependencies and manage this manually with Gradle's failOnVersionConflict() resolution strategy. We override the appropriate dependencies by forcing overrides when there are conflicts. Is Jigsaw supposed to make this easier or eliminate this problem entirely? Also, is Jigsaw supposed to replace Maven eventually?

Some of these purported issues with Jigsaw appear to be quite complex and a bit difficult for the layman Java user to understand.

> Is Jigsaw supposed to make this easier or eliminate this problem entirely?

Jigsaw totally punts on the problem of incompatible versions. In fact, it just doesn't do dependency versioning; only module versioning.

This was a fantastic, thorough, unbiased explanation.

I am going to sully it by weighing in with my opinion:

> Java packages are a fine "module" system if you're designing a single large cohesive application that has no dependencies or trust boundaries.

I've always thought that Java packages are a fine module system period. They require library authors to be prescient enough to never make backward-incompatible API changes (it's easy, just make a new package any time you want to change your API in an incompatible way).

Obviously, the open-source community has fallen down on the job in this respect, and I'm sure there is a widespread opinion that the burden of managing API changes in this way will never be met and so something like Jigsaw is necessary.

But I think the required culture change among library authors would be much preferable to adding the complexity of another module system on top of packages. And it's such an easy change for library authors to make to their behavior that there must be at least some hope of being able to reach a consensus around it.

When you think that it's mainly Oracle that's hampering Java's progress, along come IBM and Red Hat...
IBM and Red Hat are on the side of sanity on this one: why would you support a change that breaks Maven? Such a change will split Java where 'legacy' Maven-using apps will forever remain on Java 8 because of it costs money to refactor around the breaking changes.
There are so many more alternatives for the JVM these days - languages like Scala, Kotlin and Clojure - I wonder why these companies do not invest in the future and still push the old Java around.
Jigsaw is about the JVM, not the Java language. It would affect all JVM languages equally.
I too wonder why IBM would not invest in rewriting their millions of lines of code in some other language
Rewrite in Rust!
Seriously. Look at how google handles the Python2 to Python3 gap. They built a python2-go bridge and they are probably rewriting everything python2 in go. Make something expensive enough, and companies with resources will react.
The truly hilarious thing about the Python 3 switchover is that they didn't take the opportunity to remove the GIL at the same time they were making all the other breaking changes.

I guess rewriting all your code is one thing but rewriting it under the assumption of concurrency is just a bridge too far.

Or like this is post 7,345 where someone has to explain for many apps the GIL literally does nothing bad for you. Not all apps need concurrent threads in the same process.
Removing the GIL has nothing to do with making breaking changes in the language. It's also an enormously more complicated task, one which hundreds (thousands?) of developers have spent time on over a decade (two?).

I think you might be thinking that the impediment to removing the GIL is user code. It is not. The impediment is that CPython - the standard Python runtime - was designed around using the GIL, and it's extraordinarily hard to remove it and still meet the existing single-thread performance goals. There are also ancillary issues surrounding user code - particularly modules written in C, I believe - but the runtime itself is the biggest issue.

All three can already compile to JavaScript. Scala and Kotlin have early state projects that compile to native.
Because Java is a fine and mature language?
Here is maybe some reasoning behind the no votes

Concerns Regarding Jigsaw(JSR-376, Java Platform Module System) https://developer.jboss.org/blogs/scott.stark/2017/04/14/cri...

A highlight:

"Following links with enticing sounding names like “Where to start” simply gives you a bunch of links to off-site tutorials written by implementation vendors, none of which are any good. There is also a list of five different books on the topic. This developer experience is not excellent. But mostly it’s complicated because its design is genuinely enormous."

Five books? That's horrible.

(By way of introduction, I've spent a good deal of time with OSGi and none at all with JBoss or Jigsaw. I am more-or-less out of the Java environment at this point.)

Ok, here's the deal: OSGi looks big and complicated. Part of that is due to the fact that it's been around for like 20 years and part is due to people trying to do fancy things with it in the Java environment, where everyone else is trying to do fancy things with basically incompatible magic. (Virgo[1] waves desolately from off in the distance.)

[1] http://www.eclipse.org/virgo/

In reality, OSGi isn't that bad. I learned it mostly from reading (part of) the spec, which isn't the worst as far as specs go. OSGi is fundamentally a generic container; if you're familiar with servelet lifecycles and have used Tomcat to deploy and undeploy web apps, you're familiar with about 60% of OSGi, which is a subset of that. The other 40% of OSGi's fundamental complexity is built around the sole purpose of allowing an application to depend on a library A, which in turn depends on version X of library B, while at the same time the application depends on library C, which itself depends on version Y of library B. (The other 10% of OSGi's fundamental complexity is the "services", which are double-plus fun extra features that make life better in many ways. But we can ignore them here since we're talking about modularity.)

Doing that sort of thing is hard. Doing that in Java, where everything uses reflection, dynamic code generation, and classloader magic, is not so much hard as terminally bizarre.

Personally, I view a module system as an aid for the programmer: it prevents stuff that you shouldn't touch from leaking into stuff that you should touch. As a result, both OSGi (and, I think, Jigsaw) fail badly as module systems: If it waits until runtime to blow chunks, it's not helping the programmer. But, in the Java environment, modularity in my sense is impossible. (It's also unwanted, but that's a rant about Java developers for another day.)

Anyway, when you say, "oh, they're just whining that their system didn't win", you are right, sort of. But on the other hand, "their systems" have spent a long time dealing with the insane, Sorcerer's Apprentice-type nuttiness that is the rest of the Java ecosystem, and it appears that Jigsaw is just ignoring that nuttiness, that the majority of Java programmers use, and enjoy, every day. Which was the major worry from the OSGi community when the Jigsaw project was announced.

As an amusing aside: version numbers. OSGi has a moderately complicated semantic versioning [2:PDF] system to support that fundamental goal up there. OSGi version numbers look like major.minor.micro.stringy, with specific rules for bumping the major, minor, and micro numbers. Jigsaw originally started out with an incompatible version number format, apparently due to the fact that Java versions have a fixed constant major number of 1. The current [State of the module system] for Jigsaw says,

[2] http://www.osgi.org/wp-content/uploads/SemanticVersioning1.p...

> A module’s declaration does not include a version string, nor constraints upon the version strings of the modules upon which it depends. This is intentional: It is not a goal of the module system to...

I do not know of any big OSGi success apart from eclipse. At one point Spring put a lot of effort and then they dump all that at eclipse project after not seeing much traction. Apache has bunch of OSGi related projects and they do not seem very popular compared to straightforward Java/EE components.

For my applications I see some value in Jigsaw specially modular images. Despite being available for long time OSGi still is some esoteric technology without mass developer appeal whereas Jigsaw seems to have potential to be really useful to lots of developers.

It is rather sad that OSGi people keep piling on Oracle for not using their barely successful technology.

I worked a good deal with OSGi and think it is sad that it didn't get more traction in the server world. I once adapted a jetty container to use OSGi dependencies instead of packaging all dependencies into a single war file. I gained wonderfully light weight deployment units and the possibility to patch dependencies without touching downstream modules.
I used the SpringSource server (later known as Eclipse Virgo) as a sort of "platform as a service" for enterprise apps. It could deploy anything from plain war files up to OSGi-aware apps that could access custom services for configuration, db access, authentication/authorization, and so forth. Worked rather well, with decent uptime and no hair pulling on my part.

The down side was that OSGi requires modularization, which exactly no Java developers to my knowledge had any experience with. Plus OSGi modularity is entirely done at runtime and Spring's tooling to try to support it was incredibly flakey. Conversations went like:

"My app works fine on my dev box but blows chunks when I try to deploy it."

Me: "You have to be explicit about whay you are depending on in your app's manifest."

"I see your lips moving, but all i hear is barks, grunts, and squeals."

"Sigh. I'll fix it."

I can't really see jigsaw making the situation any better, especially since it explicitly ignores versioning and containers.

On the other hand you have the same kind of problems with Maven. You have to state your dependencies explicitly which can be a pain. And Maven does not support multiple versions when running tests. Everything is lumped into one classloader (afaik), where I sometimes wish Maven would make use of something OSGi like. With regards to dependency declaration both are pretty similar.
Awesome. I really like keeping two separate GUI toolkits in the docker container for my webservice. I don't print, like ever, but it's good to know i could hook up a lineprinter and do hard logs of every web request. Oh, and the CORBA and RMI registires. So comforting to know i have those options if i need them.
On the other hand, if porting our app to Java9+Jigsaw requires us to wait a year (because breakage with maven), costs like half a year to a year (I know our codebase), while upgrading our application servers (because classloading) and possibly re-doing a lot of the build process at the same time... I'll have to support Java8 for just about forever.

That upgrade wouldn't happen, or it would be really nasty. I'll rather tolerate a standard library with a lot of bollocks in there and provide security around it.

aww shucks. Yeah, i understand. Backwards compatibility is hard, and java has been pretty good (there have, of course, been a few rough spots). I'm surprised there's no answer at all for maven. I'd assumed there was some sort of

    import *
to just grab everything, like we're all doing right now. Going and adding a single line to a few hundred build.xmls doesn't sound excessive. shrug
People have been waiting a decade now? I remember listening about Jigsaw on the Java Posse back in 2009:

http://javaposse.com/java_posse_259_jigsaw_and_jsr_294_inter...

https://jcp.org/en/jsr/detail?id=294

I didn't mean wait for Jigsaw. I meant: Assume jigsaw breaks maven, i.e. it's not compatible and there's no easy ugly workaround without runtime errors. That'd mean, my place - and pretty much every place - has to wait at least a year or more, until all dependencies are ported to jigsaw. If all dependencies get ported. This sounds way too similar to the whole Python2/Python3 mess to me to be a good idea.
It would be comforting if only these classes weren't clogging my hardrive with millions of useless bytes.

Some of these libraries take more disk space than the frontpage of a news website in the browser cache!

And I feel obliged to use them sometimes lest they turn into malware out of sheer boredom.

Jigsaw is hardly the only way we can get a smaller rt.jar for containers. We already have minifiers, and they already know about rt.jar explicitly (not as a special case).

(Plus, of course, rt.jar isn't so bad if you can share it in a layer :))

> I really like keeping two separate GUI toolkits in the docker container for my webservice.

Except Jigsaw does nothing to fix this. Your webservice framework depends on jaxb which depends on activation which depends on JavaBeans (everything depends on JavaBeans). Turns out JavaBeans is in the same module as these GUI toolkits.

With the GUI toolkits its the other way around. And the beans you meant are "visual beans" (`java.awt.Component`).
>Jigsaw does not provide a clean upgrade path for Maven-based projects to move from the packaging of code into JAR files to modules.

No upgrade path from JAR files, the most common packaging system for Java?!? That sounds like a good enough reason for me to dislike it.

Jigsaw breaks far too much stuff to be useful in its present form. There are a _lot_ of cross-links between things you might expect to be separate; e.g. java.sql's date objects using java.util.Date.

Despite all the breakage, Jigsaw still totally punts on the problem of versioning. So, all it lets you do is declare that you need something (with a self-destructing opt-out); but it still doesn't solve the problem of two libraries requiring two distinct, mutually incompatible versions of the same library.

Producing clever vendoring tricks in build packages would solve actual problems that Jigsaw doesn't even begin to address. Until then, the added value seems pretty questionable.

There are a lot of nice things happening in JDK9 that just makes everything a little bit better as if by magic, and they're being held back by a standard that breaks stuff and ostensibly few people want. That doesn't mean the exercise was futile; the source code reorg is probably an improvement regardless.

I've never liked the hacks around loading multiple versions of the same library. To me, this seems fragile and bloated workaround, and it would be far better to encourage people to fix their dependencies. In other ecosystems where they can't abuse a classloader, this kind of multi-version dependency is discouraged, and at Google we completely ban it for Java.
> this kind of multi-version dependency is discouraged, and at Google we completely ban it for Java

Good to hear. It takes a surprising lot of backbone to resist the call to "just do" tortured things to fix a problem--sometimes I play the bad guy, saying "no, that's stupid, figure out a better way" but yet we see repackaged Java classes shipping here and there...

> In other ecosystems where they can't abuse a classloader

That's not abusing the classloader; it's using the classloader.

> and at Google we completely ban it for Java

Yeah, I'll sure to listen to Java tips from the developers of the Android API.

Well, we also developed Guava, so karma balanced. :)
I would be interested to learn how you cope with indirect dependencies (i.e. dependencies brought in by your dependencies). I often have to deal with the situation that both of my dependencies A and B require Y but in different versions. Neither A nor B offer versions where the dependency on Y is compatible (or the same).
That's fair, but people do have them, and they're a big part of the reason Jigsaw is suggested to be too big to fail.
Now that is a trip down memory lane. Back in 1994 I was working on a strong capabilities security model for Java with a cryptographic signature based class loader. Amongst the challenges of doing such a system was having a strong versioning system, clear and enforced lines between public and private interfaces, and a dynamic bytecode editing system which would actually elide capabilities from the class being loaded that weren't authorized (system is no good if reflection can show you things you aren't supposed to see right?)

Reading that proposal I can see that Jigsaw has run up against and is trying to 'create' some (but not all of course) very similar features on which to host a module system. And some of the very same arguments come up against them.

In particular this statement from the recommendation[1]:

Jigsaw's implementation will eventually require millions of users and authors in the Java ecosystem to face major changes to their applications and libraries, especially if they deal with services, class loading, or reflection in any way.

Is nearly a word for word echo on the reasons why my 'overly complicated and invasive' security requirements would all the people writing Java code that really didn't care all the much about such strong security guarantees.

I don't have a position one way or the other, I've been out of the Java world for a long time now, but it really struck me like Deja Vu to read that objection.

[1] https://developer.jboss.org/blogs/scott.stark/2017/04/14/cri...

Java was not publicly released until 1996? That is when I first remember it. So does this mean you were working at Sun at the time with Mr. Gosling?

One of my clients is apparently friends with Mr. Gosling and was surprised when I recognized him from a photo at a Sharks game in the "oh wow! That's James Gosling". Never met, just from having read a wikipedia page.

Yes, technically Java was announced in March of 1995. I had joined the secret project known as 'Green' in 1992 to build an OS out of it and do networking and security work. (very old picture https://stuff.mit.edu/afs/sipb/user/marc/hotjava/doc/people....)
Very cool. My very first exposure to programming Java was being given a copy of Visual J++ 1 by a neighbor along with a book. I quickly learned that real Java was better. From the time I was 14, I was in love with coding starting with GW Basic, QBasic, Turbo C++, etc.
One good reason to refuse Jigsaw: it's far far too complex.

I watched a whole hour and a half presentation on it, and the time was well necessary to explain the way things work and outline a migration path for existing code.

The worst thing about it: I can't remember much about how it was supposed to work. I just remember that it was highly unintuitive, requiring both explicit imports and exports in specific files for each module.

The fact that it is so tightly coupled to JDK refactoring also gives me pause. Seems like a design overfit. Both concerns should be handled separately.

Sure a module system can help to separate out the JDK libraries, and that's certainly a nice way to check you've achieved your objectives with the design. But when this refactoring itself is one of the major justification for the design, something is wrong.

I'm not sure that IBM and Redhat voted against current Jigsaw implementation because it's complex. It seems that it's not complex enough for them. So if we won't get Jigsaw now, we will likely get more overengineering or not get anything at all.
You are right, I don't know IBM and Red Hat reasons, but I do not trust them very far on this.

Red Hat has a great Java alternative in Ceylon with a module system which seemed far more sensible when I read about it (you should take this observations with a grain of salt and check for yourself, I'm going on hazy memories).