That seems like a pretty clunky interface. But I assume from the way that everything is spelled out explicitly that it's made to be performant. And obviously it's a big improvement over JNI.
This is the runtime API, that uses the JITs to generate the trampoline codes necessary to go from the Java calling convention to the C calling convention (upcall and downcall), a kind of like libffi for Java.
If you have a typed langages like Scala, Kotlin, Java, you can use jextract [1] which is a more high level API. jextract uses clang to parse C headers files and generates a jar file with all the C function seen as Java methods with all the glue generated for you (the downside is i believe that this jar is OS specific).
This API is more for dynamic languages like Clojure, Groovy, JRuby, where you want to do dynamic runtime calls and still get performance (as far as i understand, the JIT generates the adapter code once per call and you can reuse it).
Has P/Invoke been cross-platform for a decade? Additionally, does it parse C headers to create native type wrappers automatically? Last I checked (a long time ago), P/Invoke required you to annotate manually written prototypes?
Ah right, Mono. I keep forgetting that was a thing.
This proposal will pave the way for jextract, though. I would argue this series of JEPs is much closer in spirit to something like Rust's cxx than P/Invoke is.
No it isn't. As _old_dude_ pointed out, this is a runtime API that generates JIT wrappers. Rust's cxx does no runtime code generation, while P/Invoke does.
This time of new Java features under Oracle is such a great contrast compared to the days of Sun as the Java Steward with no funding for features. (I understand why people might have issues with Oracle, but they provide some funding).
I really think that Oracle does a great job with Java! It looks like they'd have a pretty clear roadmap with what is really important for Java in the long run. Just think about virtual threads (project Loom), or values types (project Valhalla) or GraalVM. They invested many man years in each of it and take care of considering all relevant aspects. If it is done you can trust to have a proper solution. That is the kind of technology you want to build products on!
Yes,the new memory model for instance for "immutable classes" for instance will be one of the biggest improvements for some applications as you can finally build cache line optimized data structures, which are for instance nowadays common in in-memory database systems. I think first considerations regarding this have been made already 10 years ago.
The new upcoming (or even already in preview) language and platform features also support each other very well. I think they are heading in a clear direction with the different projects.
I honestly can't tell if this is a joke or not. Valhalla is one where it was an obvious deficiency from Java 1.0, that has taken oracle about 2 decades to fix. for comparison, Microsoft was able to fix this by making "our version of Java but not completely broken" (aka C#) in less than half the time.
Except that they fully didn't, since C# 7.0, Microsoft keeps adding the System C# features into C# as means to make it more competitive with C++ workloads.
One of the biggest advantages of Java/JVM is backwards compatibility. The reason Oracle is taking so long to do Valhalla is they are doing it WHILE being able to run Java 1.0 jars.
Microsoft "fixed" things by saying "All your 2.0 net code is broken in 3.0".
no code was ever broken with .net 2.0, .net 3.0, .net 3.5, etc. .net was just mostly binary compatible, most often source compatible, but basic syntax never broke, the api/runtime often broke, but that is a different matter.
i.e. why .net core broke a lot of stuff was not syntax it was most often that libraries changed/were missing.
java's runtime was mostly source/binary compatible compared to .net, but thats why a ton of stuff was never used, like http client/serve lul..
A programming language is composed by grammar, semantics and standard library, it doesn't matter at what level it breaks, if the code or binaries cannot be brought forward without changes.
Since Oracle decided to actually remove parts of Java, they certainly heard from people using that ton of stuff that no one uses.
I don't get where you get the two decades from. Oracle acquired Sun in 2012.
The model for Java was Smalltalk, but in Smalltalk everything was an object => difficult to optimize. Primitive types were introduced in Java to optimize computation with those basic datatypes.
The state of things was fine, until Generics were introduced. Generics preserved backwards compatibility, but widened the division between object and primitive types.
This division was fine, until it started to matter that Java can't be used for high-performance applications. For enterprise application, this gap has never really mattered.
Project Loom is a game changer. At least as big as Java's original JVM w/ GC combo.
(Ya, ya, ya Java wasn't the first. But somehow Java 1.0 got mindshare. And this time Java's got a massive installed base and ecosystem, greatly increasing the likelihood for Project Loom's adoption.)
Agreed. IMO, Loom + a lot of the startup improvements will make Go somewhat obsolete compared to the JVM. The JVM developers have pretty quickly closed the gap. Where java was a poor container choice, it's become pretty nice.
If only they hadn't sued Google, Java would still have been the pre-eminent language for Android development. Sadly Android is stuck at legacy Java 8 permanently now. So, modern Java is stuck as a server-side language with dozens of competitors.
Most of the libraries of Java 11 are not available. Basic stuff like the Java 11 HTTP client. Pseudo compatibility. And this only happened because the new Gradle needed Java 11. They just did enough to make the new Gradle work. Android Java will not pass the Java 11 test suite.
They definitely messed up quite a few times. They had a lot of internal management that was down-right hostile towards open source. Not everything was rosy with Sun either though; there was immense gnawing of teeth from the community, mainly stoked by Stallman and a personal vendetta over emacs of all things.
I agree with your sentiment though. Nothing is perfect, but I do appreciate what they've been doing with GraalVM, new JEPs, and advancing the language (other than the var keyword ;)
JNI requires you to write your own C code to call into. I generally want to do that as little as possible, for many reasons (I’d have to build it and ship it myself.)
This is apparently focused on efficiently accessing off-heap memory. In Jni lot of things require copies. Only JNI API that's reliably copy-free is ByteBuffer API.
JNI is the only escape hatch in the JRE. JNA leverages JNI and so has a better API while still keeping the terrible performance. This is like JNA, but more robust, and while also being an escape hatch of its own so that you can finally get performance similar to P/Invoke when making native calls.
JNI is extremely tedious. Not only do you have to have a native library built for every platform you support, you also have to manually convert between Java and C representations of things. A lot of glue code for which there are no generators (I think). JNA is better, it still relies on JNI, but uses a tiny stub library to allow you to link to and call arbitrary functions from C dynamic libraries in pure Java.
This feature is basically JNA, but more efficient and built into the JVM itself.
Please note that this is only second incubator. After that will probably be third incubator, preview, second preview, maybe third preview. This will not land before Java 23 or so.
This can't come soon enough as far as I'm concerned. Being constrained to only memory map 2 Gb chunks of memory, with no control over their lifecycle, is extremely cumbersome and arguably one of the biggest problem with Java right now.
I'll have to check how to call `madvise` to set MADV_RANDOM as index accesses in my use case are rather random, though. But the 2Gb chunks at most have been a really big issue. Luckily, the Foreign Memory API is getting to it's final version I think. The latest changes aren't that big IMHO :-)
Yeah, tell me about it. I've built a search engine index including an on-disk hash table off MappedByteBuffers. Like chopping down a tree with a pasta sieve.
That is a part of it, but large arrays are also sliced up behind the scenes rather than kept as one contiguous segment of memory. This causes a fair bit of memory overhead, as there needs to be GC book-keeping for all the slices.
This is super interesting to know! This means that future implementations of Frozen Arrays could optimize copy-on-write operations by sharing such chunks!
This is about calling into any native operating system APIs, as long as they are callable via C or C++ (which these days means "all" operating system APIs). Think native UI controls, native hardware driver access, native access to OS services, etc.
JNI is somewhat harder to use, because you need custom glue on both sides of the border: Some custom classes in Java and some custom code on the C (and C++) side.
This proposal would remove the need for the glue on the C side and would allow a pure java solution.
Something like this has existed in third-party form for a while as JNA (https://github.com/java-native-access/jna), but now it's going to be built into the JRE itself (if the proposal passes through review)
I wrote a tiny article with a couple of examples 3 years ago. [1]
Having worked with JNI in the past, I find the FF concept convenient but still somewhat verbose. In any case, my experience is 3 years old. A lot may have changed since then.
I cannot understand why they think it is a good idea to require you to pass on the command-line the name of every module allowed to access native code. You can already do unsafe things with JNI today, that hasn't gone anywhere. And while it's much more performant than JNI was, it's still not performant enough for them to write all the syscalls in the standard library in it, unlike P/Invoke is for .NET.
> it's still not performant enough for them to write all the syscalls in the standard library in it
I'm not sure why you are inferring that. They aren't replacing openjdk vm intrinsics using this immediately, but that's hardly surprising for a brand new feature.
JNI will probably never truly die, but it's reasonable to assume that it will eventually become restricted in similar ways.
The motivation is simple: applications should become the arbiters that decide which modules get to invoke native code. Libraries don't get to nilly-willy call non-Java code anymore. If, say, a dependency upgrade introduces changes that involve a new native call, it will be very noticeable.
As painful as JNI is, it means Java has Java implementations of most things (in order to avoid JNI), so builds and distribution are a lot simpler. It's pretty common for things to just work across architectures and OSes.
On the other hand the relative difficulty of interfacing with native code may be one factor that has, for example, allowed python and go to become much more popular than java in certain areas over the last 10 years despite java arguably being a better language.
I'm waiting for Graal to release their own version of jextract that autogenerates GraalVM "@CFunction" annotations.
That would be so killer. My immediate thought was it'd be cool to write something like a Postgres native extension in a JVM language, using the Graal C FFI primitives and have Graal compile it to a native shared lib.
64 comments
[ 3.2 ms ] story [ 146 ms ] threadIf you have a typed langages like Scala, Kotlin, Java, you can use jextract [1] which is a more high level API. jextract uses clang to parse C headers files and generates a jar file with all the C function seen as Java methods with all the glue generated for you (the downside is i believe that this jar is OS specific).
This API is more for dynamic languages like Clojure, Groovy, JRuby, where you want to do dynamic runtime calls and still get performance (as far as i understand, the JIT generates the adapter code once per call and you can reuse it).
[1] https://github.com/openjdk/jextract
P/Invoke does not parse C headers, but neither does this proposal.
This proposal will pave the way for jextract, though. I would argue this series of JEPs is much closer in spirit to something like Rust's cxx than P/Invoke is.
The new upcoming (or even already in preview) language and platform features also support each other very well. I think they are heading in a clear direction with the different projects.
Span, safe stackalloc, readonly structs, bittlable structs, native delegates,....
Microsoft "fixed" things by saying "All your 2.0 net code is broken in 3.0".
no code was ever broken with .net 2.0, .net 3.0, .net 3.5, etc. .net was just mostly binary compatible, most often source compatible, but basic syntax never broke, the api/runtime often broke, but that is a different matter.
i.e. why .net core broke a lot of stuff was not syntax it was most often that libraries changed/were missing.
java's runtime was mostly source/binary compatible compared to .net, but thats why a ton of stuff was never used, like http client/serve lul..
A programming language is composed by grammar, semantics and standard library, it doesn't matter at what level it breaks, if the code or binaries cannot be brought forward without changes.
Since Oracle decided to actually remove parts of Java, they certainly heard from people using that ton of stuff that no one uses.
The model for Java was Smalltalk, but in Smalltalk everything was an object => difficult to optimize. Primitive types were introduced in Java to optimize computation with those basic datatypes.
The state of things was fine, until Generics were introduced. Generics preserved backwards compatibility, but widened the division between object and primitive types.
This division was fine, until it started to matter that Java can't be used for high-performance applications. For enterprise application, this gap has never really mattered.
(Ya, ya, ya Java wasn't the first. But somehow Java 1.0 got mindshare. And this time Java's got a massive installed base and ecosystem, greatly increasing the likelihood for Project Loom's adoption.)
Where can I read more about the version freeze?
:(
https://developer.android.com/studio/releases/gradle-plugin#...
... which up until a few months ago was the last LTS. So they're fairly up to date by now.
I agree with your sentiment though. Nothing is perfect, but I do appreciate what they've been doing with GraalVM, new JEPs, and advancing the language (other than the var keyword ;)
Can I use it to call any WinAPI without C glue?
JNA => "I write java code & load libsomething at runtime, asking for FuncName(int, int) which java can then call into native code."
You want JNA for your use case.
This feature is basically JNA, but more efficient and built into the JVM itself.
I am guessing this is for more sophisticated things than just sending data to Microsoft Excel application?
JNI is somewhat harder to use, because you need custom glue on both sides of the border: Some custom classes in Java and some custom code on the C (and C++) side.
This proposal would remove the need for the glue on the C side and would allow a pure java solution.
Something like this has existed in third-party form for a while as JNA (https://github.com/java-native-access/jna), but now it's going to be built into the JRE itself (if the proposal passes through review)
Having worked with JNI in the past, I find the FF concept convenient but still somewhat verbose. In any case, my experience is 3 years old. A lot may have changed since then.
1 - https://dev.to/petarov/java-and-native-code-in-openjdk-13-18...
I'm not sure why you are inferring that. They aren't replacing openjdk vm intrinsics using this immediately, but that's hardly surprising for a brand new feature.
The motivation is simple: applications should become the arbiters that decide which modules get to invoke native code. Libraries don't get to nilly-willy call non-Java code anymore. If, say, a dependency upgrade introduces changes that involve a new native call, it will be very noticeable.
That would be so killer. My immediate thought was it'd be cool to write something like a Postgres native extension in a JVM language, using the Graal C FFI primitives and have Graal compile it to a native shared lib.