According to the article: "A module’s declaration does not include a version string, nor constraints upon the version strings of the modules upon which it depends. This is intentional: It is not a goal of the module system to solve the version-selection problem, which is best left to build tools and container applications."
I'm surprised by this as well. I was somewhat expecting Java 9 to tackle the dependency version problem.
I'm glad they didn't. Its not essential to solve the problems the module system is designed to solve, making adoption easier. A mechanism to resolve, download and cache artifacts is still needed anyway, at least I didn't see anything in that regard.
> I'm glad they didn't. Its not essential to solve the problems the module system is designed to solve, making adoption easier. A mechanism to resolve, download and cache artifacts is still needed anyway, at least I didn't see anything in that regard.
But, had they included versioning support, they could have also built upon that later to support module distribution to finally not have to worry about maven, ivy, etc.
The main difficulty about versioning though are snapshot (development) versions that don't have a defined version number. In maven, typically these are just "*-SNAPSHOT" which may not be unique enough if two projects wanted to depend on two different snapshots of the same module.
My reading is it's hard enough as is because of backwards compatibility. They seem to be under massive pressure to deliver something for JDK 9 and can't push the deadline much longer.
If memory serves me correctly I remember Mark Reinhold saying to do it "right" they would have to run a linear equation solver like Eclipse/P2.
They say that but I don't know anybody who actually uses classpath (java -cp). Most people use a custom class loader, container and module system. In addition Oracle cops out of versioning. So I'm not sure who they are targeting.
The folks you work with seem to be more diligent than a lot of enterprise java folks I know. Explicit classpath is useful in deeply silo'd enterprise environment to get around multiple dependencies on incompatible versions of the same 3rd party jars. Often introduced while under the guns and then kept because maintenance dollars go elsewhere.
The most extreme example is probably WildFly that has just the JBoss Modules JAR in the class path. That sets up class loaders for the modular service container that in term sets up class loaders for all the deployments.
Every Java EE server (even Tomcat) sets up its own class loaders. So does Jenkins/Hudson, Maven, all RCP platforms, even applets. It's really rare to find applications where all the code is in the class path.
Java modules, which happen to exist in other languages like .NET, Delphi and Ada, are akin to having a set of dynamic libraries exposed as a single library. With the ability of having symbols that are only visible to the dynamic libraries that are part of the same module.
Except they don't mention at all how a module system should lead to faster boot times. Sure you could come up with faster lookup for classes by picking the right JAR but OSGi and JBoss Modules already do this today.
They have a protoype that does AOT compilation [1] but that's only a prototype and the gains are meager.
>Except they don't mention at all how a module system should lead to faster boot times.
They will switch from the basic jar (zip) format to a more optimized one. Additionally, with a modularized jre, it will load less modules (who use corba?). I'm quite sure there are additional reasons, but it's the 2 that I remember.
The JVM already only loads classes on demand so if you don't use Corba it doesn't have any impact on you (maybe minuscule extra seek time reading the JAR).
The jar files used to be memory mapped, and as far as I'm aware they still are. So there should be a noticeable improvement in seek time by eliminating never-used files.
You could also perhaps be looking at a situation in the future where the JRE can selectively exclude things at install time, which would be nice.
> They will switch from the basic jar (zip) format to a more optimized one.
Straight from the article:
> A modular JAR file is like an ordinary JAR file in all possible ways, except that it also includes a module-info.class file in its root directory.
> A modular JAR file can be used as a module, in which case its module-info.class file is taken to contain the module’s declaration. It can, alternatively, be placed on the ordinary class path, in which case its module-info.class file is ignored
That's why it's called a modular jar. There is also this "jmod" file described as:
"For the purpose of modularizing the Java SE Platform’s reference implementation, the JDK, we will introduce a new artifact format that goes beyond JAR files to accommodate native code, configuration files, and other kinds of data that do not fit naturally, if at all, into JAR files. This format leverages another advantage of expressing module declarations in source files and compiling them into class files, namely that class files are independent of any particular artifact format. Whether this new format, provisionally named “JMOD,” should be standardized is an open question."
"The internal files rt.jar, tools.jar, and dt.jar have been removed. The content of these files is now stored in a more efficient format in implementation-private files in the lib directory."
> For the purpose of modularizing the Java SE Platform’s reference implementation, .... Whether this new format, provisionally named “JMOD,” should be standardized is an open question.
That just talks about the JDK. There is no promise that normal applications can use this format.
OSGi isn't that complex, it's more that modularisation is inherently complex and people try to wave that away. When thinking modularisation through you quickly end up wanting depencency managament, versioning, modules and dynamic updates (meaning no restart of the VM when updating a module), and combining all of this is inherently difficult to design, and some of the stuff the JVM wasn't really designed for from the beginning. OSGi solves all that in a quite nice way, but for political reasons Oracle doesn't want to use OSGi ('not invented here syndrome').
That's a naive solution. OSGi has a very specific target of embedded applications. Thanks to its classloader, its a headache to port existing libraries which weren't written with its classloader in mind. This leads to duplicate osgi safe versions in custom artifact repos.
In the end, not a solution, but a very nice environment and development choice if you are greenfield and in the target audience.
I believe older java versions are already incompatible with newer jvms. I couldn't find it explicitly stated but i'd be surprised if 1.4 code worked 100% correct on a 1.7 jvm.
My experience so far is that the list of incompatibilities (http://www.oracle.com/technetwork/java/javase/compatibility-...) between versions makes it nearly impossible to just drop code between versions. It will run, but often implementation details have changed causing significant bugs. Your mileage may vary
I think so, the implementation is hard to control than jvms.As long as jvm can run older(e.g. 1.4) version bytecodes, jvms is compatibility, but what if jvm drop that, then support real type generic like C#, etc, that's would be wonderful.
If you watch talks like The Secret History and Tragic Fate of sun.misc.Unsafe ( https://www.youtube.com/watch?v=4HG0YQVy8UM ) you'll see that there is a move to isolate unsupported APIs and to slowly remove them once they are replaced by supported ones, but in general code that uses public APIs should continue to work without recompilation for a long time.
It will allow a group of jars to be exposed as a single entity, with internal APIs that can be exposed across jars, but not accessible outside the module.
By the time Java 10 gets out, they can see how libraries that make use of modules, reified generics, value types, GPGPU integration, JIT plugins are out of reach.
Worse even, having library devs writing two versions of their libraries.
Google's fork of Java is leading to a Python 2 - 3 scenario.
Is there an actual JSR for reified generics? I was under the impression that would generally break VM compatibility.
I have a hard time believing Google will stop supporting Java on Android, especially considering how widely they support it for other various platforms. I do know Go support for Android was recently released though.
> I have a hard time believing Google will stop supporting Java on Android
Of course they won't stop supporting it. After all they stated at Google IO 2014, that only Java matters and they could not see why anyone would want to use something else.
But they also don't show any interest to reduce the gap with the official Java.
Even the actual version lacks many of the Java 7 libraries.
> I do know Go support for Android was recently released though.
It doesn't have any official support from the Android team.
Interesting, thanks for the links. Honestly, I tend to avoid writing Java in favor of native languages at this point largely because of latency requirements. I still wonder where code sharing outweighs the type cast overhead for most JVMs though.
> It doesn't have any official support from the Android team.
Go on Android definitely seems to be in its infancy.
That is completely orthogonal. Kotlin works on top of the Java ecosystem, and therefore enjoys the same capabilities and libraries enjoyed by Java -- which Android users won't get even if they use Kotlin. Kotlin code that uses Java libraries that use features that are not available on Andorid won't work on Android.
The cool thing is that compared to many languages/platforms, Java already has an excellent module system. Just look at the horrible hacks you have to go through in C++, Ruby or Python to manage multiple projects written in different versions of the language or with different versions of dependencies are just ridiculous, especially if you're developing on multiple platforms. Oh, you want to deploy it too? Have fun.
Let alone the IDE support. I think it's truly amazing that I can make a project with 3 different JVM languages and get instant IDE autocompletion and warnings when calling methods between those languages. The source -> .class -> .jar thing has turned out to be a remarkably visionary design (or just a very lucky hit). For comparison, C# still compiles and "tools" that much slower because .class files are missing in the pipeline.
It's a testament to some of the true goodness of Java, despite its unpopularity in hipper circles, that they don't think a Maven-managed folder full of JARs is good enough.
Of course, this is really a JVM thing and you'll also get it if you use a different JVM language so you can actually get the best of both worlds.
My experience with this is a bit dated so maybe they fixed it since, but here goes: Gems are installed globally on your system. You need something like RVM to work around this, but oops, RVM doesn't work on Windows and one of your devs uses that. There's a 3-year old tutorial about how you can get it to work with cygwin, but that doesn't work well with your favourite Ruby IDE, and the list goes on. You want to deploy to a server so you try to use bundler, but bundler hard-copies a dependency called somelib-mingw-x86 into the Gemfile.lock and your linux server chokes on that. In general, the idea that bundler makes a different dependency graph depending on the OS of the dev that happens to run "bundler install" is completely beyond my understanding.
I assume that it's a bit better if all your devs (and your servers) are on the exact same OS, but compared to say NodeJS or Java, the Ruby experience is pretty messy. With Node or Java, I just say "these are my dependencies", some package manager downloads them all into a subdirectory, and then I run the app. Done. The only global dependency is the VM itself, which is typically backward compatible so no problem there, just use the latest one. C# has it just as awesome except that some libraries were written by boneheaded Windows-only people (hardcoded backslashes in pathnames, Windows Registry dependencies, stuff like that), so in practice it isn't as cool.
Practically speaking, Maven (the executable tool) and pom.xml (the language) create a hot dependency mess when projects get past the toy stage. Maven makes most simple things impossible and a few challenges trivial. "Hamstrung" is the word that comes to mind. Coupling a declarative, own-the-world, XML programming language with innumerable slap-dash/poorly maintained plugins is the pits.
All that said, maven repositories and the groupId:artifactId:version:type:classifier organization of modules are really genius. As you say, this genius shines in IDE's where, as just one example, you get nice binary/source .jar distributions automatically downloaded to your local repo. Having an IDE's debugger be able to seamlessly and accurately step through 1st party java sources and 3rd party source .jar sources is always so impressive to me.
Gradle (and Ivy before it) help paper over the pain and expose the good parts of the maven ecosystem. I have some gripes with Gradle, but after seeing it in action (and now that IDE's can grok its project configs) I'm so happy to be ditching maven/plugins and hand-written pom's for the most part.
Java needs a holistic dependency management/dependency injection/artifact repository/publishing workflow rethink where well-defined concepts like semver are enforced in the version numbering scheme, making it easier to express "safe" legal version ranges in transitive dependencies.
Because Maven (the tool) offers XML as the language, we can put any other language atop it, e.g. with https://github.com/takari/polyglot-maven we can use Clojure, Groovy, Ruby, or Scala as the language. Gradle offers only Groovy, nothing else, so we don't have any choice over how to interact with the build API.
I admit that maven on large projects can lead to some dependency problems, while not the final solution I found that Maven Enforcer Plugin[1] helps to reduce this mess, makes it harder to keep the dependencies because sometimes you have to exclude them everywhere and add your own by hand, but I don't see any better alternative, APIs change.
After I first read your last paragraph, I wondered "how might you want to see semantic versioning get enforced?" ... but I think you are instead referring to the lack of version number ranges in Maven?
58 comments
[ 2.6 ms ] story [ 88.4 ms ] threadI'm surprised by this as well. I was somewhat expecting Java 9 to tackle the dependency version problem.
But, had they included versioning support, they could have also built upon that later to support module distribution to finally not have to worry about maven, ivy, etc.
The main difficulty about versioning though are snapshot (development) versions that don't have a defined version number. In maven, typically these are just "*-SNAPSHOT" which may not be unique enough if two projects wanted to depend on two different snapshots of the same module.
If memory serves me correctly I remember Mark Reinhold saying to do it "right" they would have to run a linear equation solver like Eclipse/P2.
See also: http://wiki.osgi.org/w/images/thumb/e/e2/Classpath.jpg/500px...
Huh? That sounds like a lot of work reinventing the wheel. Could you give me some details on how that's arranged?
Every Java EE server (even Tomcat) sets up its own class loaders. So does Jenkins/Hudson, Maven, all RCP platforms, even applets. It's really rare to find applications where all the code is in the class path.
C++ modules are more like packages.
Java modules, which happen to exist in other languages like .NET, Delphi and Ada, are akin to having a set of dynamic libraries exposed as a single library. With the ability of having symbols that are only visible to the dynamic libraries that are part of the same module.
JEP 261: Module System http://openjdk.java.net/jeps/261
Java Platform Module System (JSR 376) http://openjdk.java.net/projects/jigsaw/spec/
They have a protoype that does AOT compilation [1] but that's only a prototype and the gains are meager.
[1] https://www.youtube.com/watch?v=d4B8sc7ltZk&list=PLX8CzqL3Ar...
They will switch from the basic jar (zip) format to a more optimized one. Additionally, with a modularized jre, it will load less modules (who use corba?). I'm quite sure there are additional reasons, but it's the 2 that I remember.
You could also perhaps be looking at a situation in the future where the JRE can selectively exclude things at install time, which would be nice.
Straight from the article:
> A modular JAR file is like an ordinary JAR file in all possible ways, except that it also includes a module-info.class file in its root directory.
> A modular JAR file can be used as a module, in which case its module-info.class file is taken to contain the module’s declaration. It can, alternatively, be placed on the ordinary class path, in which case its module-info.class file is ignored
So no.
"For the purpose of modularizing the Java SE Platform’s reference implementation, the JDK, we will introduce a new artifact format that goes beyond JAR files to accommodate native code, configuration files, and other kinds of data that do not fit naturally, if at all, into JAR files. This format leverages another advantage of expressing module declarations in source files and compiling them into class files, namely that class files are independent of any particular artifact format. Whether this new format, provisionally named “JMOD,” should be standardized is an open question."
Edit: from http://mreinhold.org/blog/jigsaw-modular-images
"The internal files rt.jar, tools.jar, and dt.jar have been removed. The content of these files is now stored in a more efficient format in implementation-private files in the lib directory."
That just talks about the JDK. There is no promise that normal applications can use this format.
In the end, not a solution, but a very nice environment and development choice if you are greenfield and in the target audience.
My experience so far is that the list of incompatibilities (http://www.oracle.com/technetwork/java/javase/compatibility-...) between versions makes it nearly impossible to just drop code between versions. It will run, but often implementation details have changed causing significant bugs. Your mileage may vary
It will allow a group of jars to be exposed as a single entity, with internal APIs that can be exposed across jars, but not accessible outside the module.
and http://stackoverflow.com/q/90578/309483
By the time Java 10 gets out, they can see how libraries that make use of modules, reified generics, value types, GPGPU integration, JIT plugins are out of reach.
Worse even, having library devs writing two versions of their libraries.
Google's fork of Java is leading to a Python 2 - 3 scenario.
I have a hard time believing Google will stop supporting Java on Android, especially considering how widely they support it for other various platforms. I do know Go support for Android was recently released though.
Not yet. But it might happen, it depends how value types get implemented.
Check "New Bytecodes, New Objects", "Adventures on the Road to Valhalla":
http://www.oracle.com/technetwork/java/javase/community/jlss...
> I have a hard time believing Google will stop supporting Java on Android
Of course they won't stop supporting it. After all they stated at Google IO 2014, that only Java matters and they could not see why anyone would want to use something else.
But they also don't show any interest to reduce the gap with the official Java.
Even the actual version lacks many of the Java 7 libraries.
> I do know Go support for Android was recently released though.
It doesn't have any official support from the Android team.
> It doesn't have any official support from the Android team.
Go on Android definitely seems to be in its infancy.
https://source.android.com/devices/tech/dalvik/index.html#AO...
And apparently Oracle is finally joining the third party JVMs by adding AOT support to the reference JDK, but it might only come after Java 10.
This together with the upcoming value types and JNI replacement, might eventually close the gap with the alternatives.
Not being a Java zealot, as a polyglot developer I use a bit of everything, just spreading the word.
Let alone the IDE support. I think it's truly amazing that I can make a project with 3 different JVM languages and get instant IDE autocompletion and warnings when calling methods between those languages. The source -> .class -> .jar thing has turned out to be a remarkably visionary design (or just a very lucky hit). For comparison, C# still compiles and "tools" that much slower because .class files are missing in the pipeline.
It's a testament to some of the true goodness of Java, despite its unpopularity in hipper circles, that they don't think a Maven-managed folder full of JARs is good enough.
Of course, this is really a JVM thing and you'll also get it if you use a different JVM language so you can actually get the best of both worlds.
I assume that it's a bit better if all your devs (and your servers) are on the exact same OS, but compared to say NodeJS or Java, the Ruby experience is pretty messy. With Node or Java, I just say "these are my dependencies", some package manager downloads them all into a subdirectory, and then I run the app. Done. The only global dependency is the VM itself, which is typically backward compatible so no problem there, just use the latest one. C# has it just as awesome except that some libraries were written by boneheaded Windows-only people (hardcoded backslashes in pathnames, Windows Registry dependencies, stuff like that), so in practice it isn't as cool.
All that said, maven repositories and the groupId:artifactId:version:type:classifier organization of modules are really genius. As you say, this genius shines in IDE's where, as just one example, you get nice binary/source .jar distributions automatically downloaded to your local repo. Having an IDE's debugger be able to seamlessly and accurately step through 1st party java sources and 3rd party source .jar sources is always so impressive to me.
Gradle (and Ivy before it) help paper over the pain and expose the good parts of the maven ecosystem. I have some gripes with Gradle, but after seeing it in action (and now that IDE's can grok its project configs) I'm so happy to be ditching maven/plugins and hand-written pom's for the most part.
Java needs a holistic dependency management/dependency injection/artifact repository/publishing workflow rethink where well-defined concepts like semver are enforced in the version numbering scheme, making it easier to express "safe" legal version ranges in transitive dependencies.
[1] http://maven.apache.org/enforcer/maven-enforcer-plugin/