> Should we further delay this JSR—possibly for years—in order to gain “closer consensus” by pursuing a different goal that will likely result in a design so bloated and complex that no working developer would ever use it
this.
I don't see a reason to even delay Java 9 more. It's really really really silly. Yes no solution is perfect, but most of the time when somebody want to make something perfect and actually implement everything for everybody, they fail or it ends up in super complicated stuff.
Java 10 and Java 11 looks promising, maybe more promising than 9 but 9 also has promising things and the module system might emerge into something super useful, if not - is that the end of the world? prolly not.
java has a extreme amount of bloat when it comes to releases, even more than the language itself. a lot of it are policits and still I hope that they change their "3-5 years till next release"-model soon! I mean there aren't many languages that are developed in the open and have such a aweful time frame.
the best model is prolly used by rust. smaller iterations and some other channels a la chrome.
it's really sad that the language gets so much hindered by akward polictics
There's a big difference. MSFT may take outside opinion into consideration, but they can (in general) kind of just do as they wish.
Isn't it a minor miracle that the JCP gets anything released ever? That the likes of Oracle, IBM, and a number of other mega corps that actively compete against each other in various ways can steer a committee into any agreement whatsoever is more amazing to me than the language itself.
> Red Hat Middleware initially agreed to the goals and requirements of the JSR, but then worked consistently to undermine them. They attempted to turn this JSR into something other than it was intended to be. ... I can only assume that they pursued this alternate goal in order to preserve and protect their home-grown, non-standard module system, which is little used outside of the JBoss/Wildfly ecosystem.
MR is not mincing words and I agree with his assessment.
Funny, at first I thought this had something to do with JC Penny and I was thinking "why the heck are we talking about JCP on HN". I'm glad this was something more my style.
Having said that, can some ELI5 what the module hubbub is about? I've mostly been ignoring it but now I'm curious, but as a non-Java programmer I'm not up on the debate.
Everyone has been waiting for newer Java versions forever. The companies on the community board keep waffling around and trying to add more bloat to a spec that's already wildly late.
I thought the same, and I'm slightly disappointed: I'm not sure why, but it's somehow interesting (I guess like watching a train wreck) to read about America's retail implosion.
They've been working on a module system for Java for years (codename Jigsaw), one goal of which is to modularize Java itself, and the other goal is to have a module system that developers would use. It hasn't gone very well.
Today, to run a hello world app, you have to have the entire Java SE platform on disk, including GUI code, XML parsing, CORBA, a MIDI player, etc. A lot of that code is itself written in Java; you'll find there's a 64MB "rt.jar" file that provides a lot of stuff. One goal of Jigsaw is to allow you to write smaller Java applications on memory-constrained devices.
Jigsaw does an OK job at modularizing Java, but has mostly failed at its other goal: to provide a standard module system that Java developers would use. I'm sure somebody's excited to use Jigsaw in their application code, but not many.
For example, Jigsaw's module system has no notion of libraries with versions. Imagine using Maven or Gradle without any version numbers. That's Jigsaw.
Jigsaw also set a bunch of other rules that get broken a lot in practice: no circular dependencies, immutable dependencies declared up front, resolve all modules eagerly, etc. These are good guidelines, but it may not be appropriate to make them iron clad rules.
In contrast, there's a hugely complicated monster module system implemented in Java userland, OSGi. OSGi has a ton of features, including letting you load multiple versions of the "same" jar at once, so one part of your app can use the old version while another part uses a newer version. (And then there's Maven, Gradle, etc.)
This certainly wouldn't be the first time Java tried to deliver a "standard" solution for a problem that we used to solve in userland that went ignored by Java's dev community. (Logging?!) If we all ignored Jigsaw and kept using Maven/Gradle/OSGi, but the Java platform became more modular, that's probably a win.
But Jigsaw includes another poisonous feature; it "improves security" by forbidding access to internal JDK APIs like sun.misc.Unsafe, which provides access to direct memory management. For better or worse, that internal API has been used and abused by a lot of popular Java infrastructure libraries, so they're now introducing a new Unsafe feature in Java 9, but it's not the same one, "breaking" compatibility. It's their reserved right to break that API, but but but.
And finally there's the fact that Java 9 has taken a hugely long time, and this is significantly due to Jigsaw. If Jigsaw misses Java 9, does it land in Java 10? Throw it out completely? (Does Java ever get modularized?)
I haven't followed the process, but if they just wanted to break up rt.jar, why didn't they do just that and break it into smaller jars that you may/may not include on the classpath?
That's pretty much what they've done. Java modules are just JARs with an additional file in them that states what they depend on, along some other stuff like which packages are meant to be used and which are internal.
To figure out the set of JARs you need, you need to specify which JARs your program needs, and then follow the dependencies. Which is what the new jlink tool does.
Restricting access by default to internal APIs isn't necessarily a part of Jigsaw. Any app with a security manager installed had such restrictions too, because internal APIs can be used to bypass permissions.
And, sun.misc.Unsafe isn't actually restricted. Plenty of other internal APIs are but they did actually look at what apps were using the most frequently and not restrict them: you can still access Unsafe (or most of it at least) in java 9 apps (and any internal API by just setting some command line flags).
It's similar to pre-ES6 JavaScript, the language doesn't have it's own concept of modules so a bunch of different "solutions" emerged and the entire ecosystem fractured because even though you were just trying to write a leftpad package you had to decide on one scheme and whoever wanted to use it better used the same one, too.
But this is Java, so everything is extra stupid. I can't say I have had much experience with Maven but I will say that what little bits I saw left me wanting to nuke it's creators from this planet. Even with nothing to do you can trust the Maven build to take an hour as it very slowly polls a bunch of online repositories and then does a bunch more tasks all for naught.
Then there is the OSGi system, but that is a synonym for "Eclipse" and all the cool kids want nothing to do with that, it's one wholly closed ghetto. Can't blame them, the x0.01 engineers that designed OSGi figured a mere module system is beneath them, so they integrated what seems to be a half-baked dependency injection system based on runtime-checked XML files. They figured type checking is holding them back. Also the tool support is entirely absent unless you are willing to use a null pointer exception throwing UI abomination in Eclipse.
My primary experience with Gradle has been in the context of Android and I figured people that come up with a build system where you can't go back and build software a mere 3 months old should really find a different job.
Maven takes a long time on every new build because it downloads all the dependencies to cache.
If you aren't a regular Java developer, and you pull a simple GitHub project and build it, it might need download the dependencies, source code, and docs (if the build is configured as such).
The next time you run, it will be cached and finish quickly.
No different than one of your first npm installs .
I never understood the need for the JCP to build a whole new standard for modularization. Why didn't they rubber stamp the OSGi spec which is used everywhere, is extremely solid, has multiple great implementations and provides stuff that this standard not even has (like runtime updating of modules, versioning and more). Felt to me like politics/not invented here syndrome from the start, no wonder there is so much pushback from the community about this standard.
Modularization itself is complex.
No matter if OSGi, Jigsaw or whatever, modularization done right is non trivial. I wonder when ppl. start realizing that instead of blaming it on implementations.
OSGi is probably the worst designed piece of technology I've ever dealt with.
If all you want is basic plugins it is horrifically complex, obtuse and unwieldy. Which is why very few people can be bothered dealing with it and just stick with Service Loaders.
I think that if this JSR doesn't get the requisite votes, Oracle will release it anyway and use their ownership of the Java trademark to terminate the JCP, disband the JCP Executive Committee, and make it just another Oracle product. Maybe it'll even be totally proprietary without an OpenJDK release.
Basically, Red Hat and IBM get to choose whether or not Java gets closed up again based on how they vote.
What worries me is that Red Hat and IBM are handing Oracle the perfect excuse to close up Java on a silver platter.
Java 9 is very late in development, it's scheduled for release in a month, and it would have already been released by now if Oracle hadn't held it back for additional polish. For Red Hat and IBM to pull this now is a violation of the JCP's spirit, and it makes the entire Java Community Process look bad. It's 100% in bad faith, and it's a huge black eye for the JCP.
I wouldn't be surprised one iota if Oracle uses this as an excuse to pull the plug on the JCP and declares that Java will solely be an Oracle product from now on developed solely by Oracle employees without any third-party contributions. Maybe they'll even use the death of the JCP to pull the plug on OpenJDK and make Java 100% proprietary again.
Honestly, if Oracle does kill the JCP, I'd be fine with that. I'm just afraid for the future of OpenJDK.
Could the OpenJDK folk just say, "fuck off" and do their own thing? Basically break Java into, I don't know, Fresh Press and Orcalse Brew? All of the old IP made available to OpenJDK and new stuff being their own.
OpenJDK is GPLd, so anyone can fork it or continue development on their own, although they might lose the right to use the trademarked word Java to describe it. But they might not be able to support future Oracle Java versions and features, because Oracle wouldn't release the source for those under a liberal license and would probably sue any clean-room reimplementers under the "copyrighted API" legal theory that they got upheld against Google.
The runtime behavior (GC, JIT, etc) would also diverge, since the optimizations, memory layout, etc. are not covered by a spec you can try to implement exactly. Today complex programs often take JVM implementation and optimization details into account (and sometimes can or have to be optimized for particular JVM versions, since e.g. the available GC algorithms have changed several times).
Finally, some of those OpenJDK folk are Oracle employees, and they would leave. Even if IBM and RedHat and everyone else united against Oracle, which is uncertain at best (see: IBM's history with Apache Harmony), it would take a lot of new money & effort being poured in just to match Oracle Java development every version.
And even if the new free fork was as good as Oracle Java, but not significantly better (technology and/or support wise), the market probably wouldn't shift. Today very many companies run Oracle Java and not OpenJDK Java on their production servers, even though they don't have Oracle support contracts, because there's a perception that Oracle Java is better or more stable.
There are various other problems I can imagine. Maybe Oracle isn't obligated to keep providing the TCK for new Java versions to OpenJDK and its descendants. Sun licensed it to OpenJDK, but maybe Oracle could find a way around that, or just stop publishing the TCK at all and make sure it was in practice really hard to be 100% compatible with future Java.
Sure they could. But in the long run that fragmentation would be the death of the Java ecosystem. None of the application vendors and open source projects (including alternative JVM languages like Clojure and Scala) would have a viable platform to target anymore. Everything would be perpetually stuck at Java SE 8 or 9 until either one of the major players gains a dominant position, or (more likely) all of the developers and users just gradually migrate to other platforms.
Without the JCP making Java a "standard" language, Java no longer meets the selection criteria for many government organizations for a programming language. Java is only usable by Oracle to write their own products if they let the JCP die.
JBI déjà vu. I see some parallelism between this JSR and the Java Business Integration (JBI, JSR-208) JSR back in 2005.
Instead of building a standard for integration, they ended up specifying a standard for interoperability between integration components from different vendors, i.e. a "meta"-integration system (using the same term as the author).
The result was a spec that allowed an (a) HTTP adapter from Oracle to exchange data with a (b) File adapter from IBM – through (c) an ESB implemented by TIBCO, for example. Essentially something that nobody would even think of deploying (in the era of proprietary, expensive and lock-in middleware).
Obviously the standard was DOA and had very little traction. Not even the JSR members implemented it. The only memorable implementation was Apache ServiceMix, which later transitioned to OSGi, spinning off its kernel as Apache Karaf.
In hindsight, the vendors might have acted like trojan horses with JBI, steering the spec into a territory that would be ultimately worthless.
The lesson to learn is that commercial vendors don't gain anything by standardising on those elements they consider strategic differentiators (like their module system).
While I don't have good knowledge about modules, I can only say, that I don't want Java SE to turn into Java EE. Let complex things stay complex, but SE should stay simple if possible. If modules were enough to modularize JRE itself, they are good enough to ship, modularizing JRE is a huge feat right now. They need to leave enough points to extend later, of course, so eventually everyone will be happy. All decisions set in stone now should be good, and that's enough. Everything that could be changed later without breaking compatibility could be far from perfect, I can live with it.
Java SE is already extremely complex. I'll bet the total cyclomatic complexity metric for Open JDK is now higher than any Java EE implementation. So if you're looking for simplicity it's too late; that ship sailed years ago.
And complexity isn't necessarily a bad thing if we get real value out of it.
Discarding unneeded Java SE modules is just a minor optimization for packaging and deployment. It doesn't do anything to reduce platform complexity; rather the opposite in fact.
I guess that depends how you define complexity. Jigsaw modularised the Java platform itself, and so you can now write an app that uses only java.base and everything else will be discarded by jlink. Surely such a stripped image is in some real way less complex than a full JDK?
I'd say Jigsaw does simplify some things (though at the cost of additional complexity elsewhere) - for instance, classpath conflicts are a constant source of pain in large projects and the JVM will now detect them at startup. You can hide internal code from users now, which if nothing else will shrink the amount of stuff that appears in IDE autocompletion and javadocs. It's also a security improvement.
The stripped image will be slightly smaller and that makes it marginally easier to deploy to resource-limited devices. But adding that feature increased the overall complexity of the Java SE platform.
Ultimately this is about who decides about Java. The community or Oracle. Is it an open process where the users have a say or are they just forced to obey authority? Red Hat and IBM, weird as it may seem, decided against ordre de Mufti.
It IS an open process which is why we are reading articles like this and have seen equivalents from Red Hat and Oracle. Everything has been transparent and open.
And you can name drop companies all you like but this is purely about the design and implementation. And frankly the meta module approach seams ludicrous to me.
Agree - the meta model idea is exactly the kind of nonsense that people make fun of Java for - complexity on top of complexity. That doesn't mean that Jigsaw doesn't have room for improvement though.
Redhat is a commercial company with commercial interests just like IBM and Oracle. It's not the community.
'The community' is a non structured dynamic open source ecosystem that dwarfs Redhat or any single company. No one can claim to speak for the 'community' when no such overarching organized structure exists.
Various Redhat interests go around online discussion bullying others projects and open source companies by claiming to speak for the 'community'. This is just self serving and has to stop. They are speaking for themselves and their interests.
Years ago, I was part of the team trying to define CORBA. While CORBA would go on to be a cautionary tale for bloated, useless meta-specifications (we would blackly joke that it was the only specification that made things less specific), once upon a time it was a relatively simple, useful, and elegant concept. Watching the industry consensus process destroy it was heartbreaking and educational.
Lesson: There can be no consensus when members of a group are for-profit corporations with selfish interests and zero interest in the common good, there can only be lowest common denominator.
> There can be no consensus when members of a group are for-profit corporations with selfish interests and zero interest in the common good, there can only be lowest common denominator.
reply
It is really sad that they focused on access control and not on distribution, respositories, versioning, compilation and runtime support like we wanted to do the first time around in JSR-277.
Those things are already handled pretty well by Maven and Gradle though. The thing those systems can't do / don't try to do is encapsulation, which ideally needs JVM support. OSGi tried to do it all with classloaders but it's hardly used and that doesn't help at compile time. Ditto for modularising the JDK itself.
Jigsaw doesn't try to compete with the existing build tools. That's not necessarily a bad thing.
I would have liked them to standardize the way Maven/Gradle works and have support in the compiler and source code for them. Kind of like how Joda time inspired the Java 8 time libraries. Encapsulation could have been handled by a Java agent. My guess is that it will have a bunch of bugs in it anyway that allows you to get around it just like the applet sandbox.
Build tools would still have a place in the world for things unrelated to dependencies and compilation. Being able to write code that uses dependent modules without using some third party tool would drastically simplify using Java. As it stands things like JShell are basically useless without support like this.
51 comments
[ 2.7 ms ] story [ 102 ms ] threadthis. I don't see a reason to even delay Java 9 more. It's really really really silly. Yes no solution is perfect, but most of the time when somebody want to make something perfect and actually implement everything for everybody, they fail or it ends up in super complicated stuff. Java 10 and Java 11 looks promising, maybe more promising than 9 but 9 also has promising things and the module system might emerge into something super useful, if not - is that the end of the world? prolly not.
java has a extreme amount of bloat when it comes to releases, even more than the language itself. a lot of it are policits and still I hope that they change their "3-5 years till next release"-model soon! I mean there aren't many languages that are developed in the open and have such a aweful time frame. the best model is prolly used by rust. smaller iterations and some other channels a la chrome. it's really sad that the language gets so much hindered by akward polictics
Isn't it a minor miracle that the JCP gets anything released ever? That the likes of Oracle, IBM, and a number of other mega corps that actively compete against each other in various ways can steer a committee into any agreement whatsoever is more amazing to me than the language itself.
MR is not mincing words and I agree with his assessment.
Having said that, can some ELI5 what the module hubbub is about? I've mostly been ignoring it but now I'm curious, but as a non-Java programmer I'm not up on the debate.
There was actually a lot of talk about JC Penny on HN when Ron Johnson took over
Today, to run a hello world app, you have to have the entire Java SE platform on disk, including GUI code, XML parsing, CORBA, a MIDI player, etc. A lot of that code is itself written in Java; you'll find there's a 64MB "rt.jar" file that provides a lot of stuff. One goal of Jigsaw is to allow you to write smaller Java applications on memory-constrained devices.
Jigsaw does an OK job at modularizing Java, but has mostly failed at its other goal: to provide a standard module system that Java developers would use. I'm sure somebody's excited to use Jigsaw in their application code, but not many.
For example, Jigsaw's module system has no notion of libraries with versions. Imagine using Maven or Gradle without any version numbers. That's Jigsaw.
Jigsaw also set a bunch of other rules that get broken a lot in practice: no circular dependencies, immutable dependencies declared up front, resolve all modules eagerly, etc. These are good guidelines, but it may not be appropriate to make them iron clad rules.
In contrast, there's a hugely complicated monster module system implemented in Java userland, OSGi. OSGi has a ton of features, including letting you load multiple versions of the "same" jar at once, so one part of your app can use the old version while another part uses a newer version. (And then there's Maven, Gradle, etc.)
This certainly wouldn't be the first time Java tried to deliver a "standard" solution for a problem that we used to solve in userland that went ignored by Java's dev community. (Logging?!) If we all ignored Jigsaw and kept using Maven/Gradle/OSGi, but the Java platform became more modular, that's probably a win.
But Jigsaw includes another poisonous feature; it "improves security" by forbidding access to internal JDK APIs like sun.misc.Unsafe, which provides access to direct memory management. For better or worse, that internal API has been used and abused by a lot of popular Java infrastructure libraries, so they're now introducing a new Unsafe feature in Java 9, but it's not the same one, "breaking" compatibility. It's their reserved right to break that API, but but but.
And finally there's the fact that Java 9 has taken a hugely long time, and this is significantly due to Jigsaw. If Jigsaw misses Java 9, does it land in Java 10? Throw it out completely? (Does Java ever get modularized?)
Look at this module graph:
http://download.java.net/java/jdk9/docs/api/javafx.web-summa...
To figure out the set of JARs you need, you need to specify which JARs your program needs, and then follow the dependencies. Which is what the new jlink tool does.
And, sun.misc.Unsafe isn't actually restricted. Plenty of other internal APIs are but they did actually look at what apps were using the most frequently and not restrict them: you can still access Unsafe (or most of it at least) in java 9 apps (and any internal API by just setting some command line flags).
But this is Java, so everything is extra stupid. I can't say I have had much experience with Maven but I will say that what little bits I saw left me wanting to nuke it's creators from this planet. Even with nothing to do you can trust the Maven build to take an hour as it very slowly polls a bunch of online repositories and then does a bunch more tasks all for naught.
Then there is the OSGi system, but that is a synonym for "Eclipse" and all the cool kids want nothing to do with that, it's one wholly closed ghetto. Can't blame them, the x0.01 engineers that designed OSGi figured a mere module system is beneath them, so they integrated what seems to be a half-baked dependency injection system based on runtime-checked XML files. They figured type checking is holding them back. Also the tool support is entirely absent unless you are willing to use a null pointer exception throwing UI abomination in Eclipse.
My primary experience with Gradle has been in the context of Android and I figured people that come up with a build system where you can't go back and build software a mere 3 months old should really find a different job.
If you aren't a regular Java developer, and you pull a simple GitHub project and build it, it might need download the dependencies, source code, and docs (if the build is configured as such).
The next time you run, it will be cached and finish quickly.
No different than one of your first npm installs .
Sure modules of this kind are a bit more complicated but they shouldn't be significantly so. Often the JCP tends to over-engineer and over-complicate.
If all you want is basic plugins it is horrifically complex, obtuse and unwieldy. Which is why very few people can be bothered dealing with it and just stick with Service Loaders.
Why must every advancement go through committee?
Basically, Red Hat and IBM get to choose whether or not Java gets closed up again based on how they vote.
Java 9 is very late in development, it's scheduled for release in a month, and it would have already been released by now if Oracle hadn't held it back for additional polish. For Red Hat and IBM to pull this now is a violation of the JCP's spirit, and it makes the entire Java Community Process look bad. It's 100% in bad faith, and it's a huge black eye for the JCP.
I wouldn't be surprised one iota if Oracle uses this as an excuse to pull the plug on the JCP and declares that Java will solely be an Oracle product from now on developed solely by Oracle employees without any third-party contributions. Maybe they'll even use the death of the JCP to pull the plug on OpenJDK and make Java 100% proprietary again.
Honestly, if Oracle does kill the JCP, I'd be fine with that. I'm just afraid for the future of OpenJDK.
The runtime behavior (GC, JIT, etc) would also diverge, since the optimizations, memory layout, etc. are not covered by a spec you can try to implement exactly. Today complex programs often take JVM implementation and optimization details into account (and sometimes can or have to be optimized for particular JVM versions, since e.g. the available GC algorithms have changed several times).
Finally, some of those OpenJDK folk are Oracle employees, and they would leave. Even if IBM and RedHat and everyone else united against Oracle, which is uncertain at best (see: IBM's history with Apache Harmony), it would take a lot of new money & effort being poured in just to match Oracle Java development every version.
And even if the new free fork was as good as Oracle Java, but not significantly better (technology and/or support wise), the market probably wouldn't shift. Today very many companies run Oracle Java and not OpenJDK Java on their production servers, even though they don't have Oracle support contracts, because there's a perception that Oracle Java is better or more stable.
There are various other problems I can imagine. Maybe Oracle isn't obligated to keep providing the TCK for new Java versions to OpenJDK and its descendants. Sun licensed it to OpenJDK, but maybe Oracle could find a way around that, or just stop publishing the TCK at all and make sure it was in practice really hard to be 100% compatible with future Java.
Instead of building a standard for integration, they ended up specifying a standard for interoperability between integration components from different vendors, i.e. a "meta"-integration system (using the same term as the author).
The result was a spec that allowed an (a) HTTP adapter from Oracle to exchange data with a (b) File adapter from IBM – through (c) an ESB implemented by TIBCO, for example. Essentially something that nobody would even think of deploying (in the era of proprietary, expensive and lock-in middleware).
Obviously the standard was DOA and had very little traction. Not even the JSR members implemented it. The only memorable implementation was Apache ServiceMix, which later transitioned to OSGi, spinning off its kernel as Apache Karaf.
In hindsight, the vendors might have acted like trojan horses with JBI, steering the spec into a territory that would be ultimately worthless.
The lesson to learn is that commercial vendors don't gain anything by standardising on those elements they consider strategic differentiators (like their module system).
EDIT: minor edits for clarity.
And complexity isn't necessarily a bad thing if we get real value out of it.
I'd say Jigsaw does simplify some things (though at the cost of additional complexity elsewhere) - for instance, classpath conflicts are a constant source of pain in large projects and the JVM will now detect them at startup. You can hide internal code from users now, which if nothing else will shrink the amount of stuff that appears in IDE autocompletion and javadocs. It's also a security improvement.
It IS an open process which is why we are reading articles like this and have seen equivalents from Red Hat and Oracle. Everything has been transparent and open.
And you can name drop companies all you like but this is purely about the design and implementation. And frankly the meta module approach seams ludicrous to me.
'The community' is a non structured dynamic open source ecosystem that dwarfs Redhat or any single company. No one can claim to speak for the 'community' when no such overarching organized structure exists.
Various Redhat interests go around online discussion bullying others projects and open source companies by claiming to speak for the 'community'. This is just self serving and has to stop. They are speaking for themselves and their interests.
Lesson: There can be no consensus when members of a group are for-profit corporations with selfish interests and zero interest in the common good, there can only be lowest common denominator.
This 100 times.
Jigsaw doesn't try to compete with the existing build tools. That's not necessarily a bad thing.
Build tools would still have a place in the world for things unrelated to dependencies and compilation. Being able to write code that uses dependent modules without using some third party tool would drastically simplify using Java. As it stands things like JShell are basically useless without support like this.