>The single license annual subscription price for Zing is $3500 USD per server, with significantly lower prices for higher volumes and longer-term subscriptions.
From where? They aren't making Alphas anymore, or GaN 5GHz LN2 cooled monsters. At this point SiC is the material as power dissapation has become the limiting factor.
You can load balance your app for 3.5k $ on additional server. By reducing load, time spend in GC will be less. Additionality by having more hardware you can use more aggressive strategy[0].
I have taken Zing for a spin on 24 c3.8xl ec2 instances running Elasticsearch with 32G heaps and just under 2 billion indexed documents with millions of updates/day. The results are considerably better. If I just went up to 30 or even 50 ec2 instances on HotSpot/Openjdk, I would see no performance benefit with regard to GC. And I have performed this test. I'm not saying dont use Java because of GC, but I am saying don't overestimate the value of more efficient garbage collection in any case.
Shoutout to the Azul guys, Simon, Elaine and Paul for coming on-site and watch me compile all their extensions on Amazon Linux and help me perform all the testing.
Besides the benefits the new JIT provides, Zing will help you if your workload runs with / consumes over 8GB of RAM. At this point, HotSpot GC pauses [1] starts becoming noticeable (i.e. jitter).
I have no idea if HotSpot optimises the specific loop Gil posted, but I've seen Intel contribute a lot of upgrades to use AVX2 and AVX512, like optimised compression intrinsics. And as noted, LLVM is not really designed for Java and they had to spend three years putting in optimisations that Java-specific JITCs have had for a long time. How many contributions to LLVM will be directly applicable to Java, given that all the other contributors are focused on C++-style languages? Use of new instructions, sure, in some cases, but they aren't going to contribute intrinsics for the Java APIs.
ReadyNow and C4 are very cool. Java has needed these for a long time and Zing users have been benefiting for a long time. However, Java 9 gets AOT compilation for some cases, which gives at least some of the benefit of ReadyNow (there is still warmup time, as HotSpot AOT does not cache profiles and re-profiles each time, but you get a lot closer to peak performance a lot faster). And Shenandoah is driving down pause times for huge heaps, which is where C4 shines.
That said, Java 9 has not yet shipped, and Zing has. So congrats to Gil Tene and his team. The competition between Zing and HotSpot only helps.
If you're testing with JDK8 you're missing the last two years of work on the optimiser. I'd be interested to see what Java 9 does, as it's (supposedly) coming out in a few months.
I'm in no way arguing that Intel and others do not also contribute to HotSpot. It's just that MUCH more attention and seems to be paid to LLVM, and the amount and impact of contributions there are bigger.
As to the AVX2 work contributed: there are all sorts of "easier to integrate" AVX2 uses that had been added to HotSpot over time (especially with intrinsics, which tend to be nice and isolated from the rest of the code gen), but the vectorizer in C2 is not nearly as capable as the one in LLVM, and the results speak for themselves. And I expect o see a similar gap open when AVX512 becomes available on Skylake EPs soon.
On the "how many contributions directly applicable to Java vs. C++ question": we were surprised at how readily those "C++ focused things" apply to idiomatic java code. The LLVM contributions can probably be better describes as "expecting certain patterns and idioms that are common in C/C++". And once you bring the Java code to LLVM IR form and "remove the breaks" as much as possible (e.g. fix how things like safepoints, GC barriers, and depot points used to defeat most optimizations), the patterns seem to be there in the Java code as well. The vectorization u-bench thing here is certainly a good example of that in play. We didn't massage it to find the vectorization opportunity. I didn't even get specific directions or tips on "write it this way". I just wrote some "I think this loop should be vectorizeable with AVX2" pieces of java code and the existing vectorizer seemed to have just picked them up and had its way with them.
Yes, I can well believe that LLVM has put a lot more effort into auto-vectorisation than HotSpot has, given the relatively different ways in which Java and C++ tend to be used. And I don't see much effort put into auto-vectorisation in Graal either, from my occasional browsings of the commit logs. The research in HotSpot JIT compilers seems to be focused more on compiling more dynamic languages like JavaScript and Ruby faster, rather than C++ style numeric code: I guess reflecting a focus on business apps that probably don't contain many hot array loops relative to scripting language use.
I look forward to benchmarks. The blog post contains a graph that purports to compare HotSpot vs Zing but there is no Y axis, so I guess it's meant for illustration of the basic idea than actual performance comparisons.
While optimization for dynamic languages is a cool research area for a JIT compiler, and may one day help expand the use of dynamic languages on the JVM, the VAST majority of code that runs on JVMs in written in Java, Scala, and Clojure, none of which is dynamic. What you do see in those languages is plenty of idiomatic loops, stream, and buffer processing. All of which benefit greatly from the new vectorization capabilities in AVX2 (and the soon to be widely available AVX512).
perhaps someone should advertise AVX2 and AVX512 under the tag "It's not you father's numeric-processing-focused vectorization"
As for the charts in the blog post: yes, it's a depiction. The Y axis gains will vary by application, obviously. And we've seen that gain range form a few % to high 10s of %s with real workloads, measured by real customers who use them. Use cases span a wide range, including databases, in-memory query engines, search and indexing, protocol gateways, stream processing, messaging, analytics, and matching engines. None are things that classically make you think "numeric processing", BTW.
Yes, I'd very much appreciate some sort of blog post or link on how AVX2/512 can accelerate ordinary business type apps. I know it can be used to accelerate things like hashing, checksumming and so on, but if it's widely applicable to the sort of code normally found on the JVM, that could make a very interesting set of articles.
I think in the current form that's unlikely. It's Linux only and needs to be built on the exact same CPU. This essentially means that you have to compile on the same system as you execute.
29 comments
[ 88.2 ms ] story [ 1205 ms ] threadWas that the dis I think it was?
>The single license annual subscription price for Zing is $3500 USD per server, with significantly lower prices for higher volumes and longer-term subscriptions.
[0]http://www.cakesolutions.net/teamblogs/low-pause-gc-on-the-j...
Shoutout to the Azul guys, Simon, Elaine and Paul for coming on-site and watch me compile all their extensions on Amazon Linux and help me perform all the testing.
Besides the benefits the new JIT provides, Zing will help you if your workload runs with / consumes over 8GB of RAM. At this point, HotSpot GC pauses [1] starts becoming noticeable (i.e. jitter).
Notes:
[1] - http://www.azul.com/resources/azul-technology/zing-consisten...
The post makes it sound like Intel only contributes patches for new instruction sets to LLVM. But they are also active HotSpot contributors.
For instance, here's a post from an Intel employee contributing optimisations to use AVX512:
http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/...
I have no idea if HotSpot optimises the specific loop Gil posted, but I've seen Intel contribute a lot of upgrades to use AVX2 and AVX512, like optimised compression intrinsics. And as noted, LLVM is not really designed for Java and they had to spend three years putting in optimisations that Java-specific JITCs have had for a long time. How many contributions to LLVM will be directly applicable to Java, given that all the other contributors are focused on C++-style languages? Use of new instructions, sure, in some cases, but they aren't going to contribute intrinsics for the Java APIs.
ReadyNow and C4 are very cool. Java has needed these for a long time and Zing users have been benefiting for a long time. However, Java 9 gets AOT compilation for some cases, which gives at least some of the benefit of ReadyNow (there is still warmup time, as HotSpot AOT does not cache profiles and re-profiles each time, but you get a lot closer to peak performance a lot faster). And Shenandoah is driving down pause times for huge heaps, which is where C4 shines.
That said, Java 9 has not yet shipped, and Zing has. So congrats to Gil Tene and his team. The competition between Zing and HotSpot only helps.
I'm actually curious to see what the output assembly of LLVM is for that loop, because JDK8 does not optimize a branch in a loop.
But the JVM does do a pretty decent job using ymm<n> registers if you write the loops without branches.
But you end up with really obscure java code like
"(((a - b) ^ (b - a)) >>> 63) ^ 1"
To convert a branch into an integer.
Enjoy!
As to the AVX2 work contributed: there are all sorts of "easier to integrate" AVX2 uses that had been added to HotSpot over time (especially with intrinsics, which tend to be nice and isolated from the rest of the code gen), but the vectorizer in C2 is not nearly as capable as the one in LLVM, and the results speak for themselves. And I expect o see a similar gap open when AVX512 becomes available on Skylake EPs soon.
On the "how many contributions directly applicable to Java vs. C++ question": we were surprised at how readily those "C++ focused things" apply to idiomatic java code. The LLVM contributions can probably be better describes as "expecting certain patterns and idioms that are common in C/C++". And once you bring the Java code to LLVM IR form and "remove the breaks" as much as possible (e.g. fix how things like safepoints, GC barriers, and depot points used to defeat most optimizations), the patterns seem to be there in the Java code as well. The vectorization u-bench thing here is certainly a good example of that in play. We didn't massage it to find the vectorization opportunity. I didn't even get specific directions or tips on "write it this way". I just wrote some "I think this loop should be vectorizeable with AVX2" pieces of java code and the existing vectorizer seemed to have just picked them up and had its way with them.
Yes, I can well believe that LLVM has put a lot more effort into auto-vectorisation than HotSpot has, given the relatively different ways in which Java and C++ tend to be used. And I don't see much effort put into auto-vectorisation in Graal either, from my occasional browsings of the commit logs. The research in HotSpot JIT compilers seems to be focused more on compiling more dynamic languages like JavaScript and Ruby faster, rather than C++ style numeric code: I guess reflecting a focus on business apps that probably don't contain many hot array loops relative to scripting language use.
I look forward to benchmarks. The blog post contains a graph that purports to compare HotSpot vs Zing but there is no Y axis, so I guess it's meant for illustration of the basic idea than actual performance comparisons.
perhaps someone should advertise AVX2 and AVX512 under the tag "It's not you father's numeric-processing-focused vectorization"
As for the charts in the blog post: yes, it's a depiction. The Y axis gains will vary by application, obviously. And we've seen that gain range form a few % to high 10s of %s with real workloads, measured by real customers who use them. Use cases span a wide range, including databases, in-memory query engines, search and indexing, protocol gateways, stream processing, messaging, analytics, and matching engines. None are things that classically make you think "numeric processing", BTW.
Thanks for the followup!
It would be nice if there was a developer version, e.g. not for commercial server usage, but as your local/desktop JVM installation.