I know that benchmarks are damned lies, but I'm honestly surprised it's this close. Dart isn't even at 1.0 yet, and the JVM is almost 20 years old with millions (billions?) of dollars into optimizing it.
And to top it all off, the dart vm throws away static types when it hits the vm - all of this is done through its tracing jit (just like v8).
It is not about maturity - I am pretty sure first version of C# was performing better then Java back then, because C# team could learn from Java mistakes.
Neither V8 nor Dart are tracing JITs. They're method-at-a-time JITs just like the JVM. They just have to do more extensive type-feedback optimizations than the JVM because the language are dynamically typed.
I'm not sure of DeltaBlue or the Computer Language Benchmarks Game is more representative of "average programs", but they appear to give very different results in their comparisons. On the Benchmarks Game dart is crushed: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
I guess people will have to decide which results are more likely to reflect the performance of their applications. But the extreme nature of the difference is interesting in and of itself.
P.S. I have not looked at the Java code to see if it is idiomatic Java either- And this really make be want to port DeltaBlue to Go...
I had a quick look at this. It appears the DartVM is now beating the JVM even at 100,000 iterations (Previously Java edged out Dart at higher iterations).
100,000 iterations takes ~45 seconds to run on my computer.
At a lower number of iterations the DartVM beats the JVM by a significant margin.
The benchmark is easy to run (On linux, maybe macos too). Just check it out, make sure dart, java and gcc are in your path, and execute run.sh.
Note, this code doesn't use the benchmark harness because I wanted to include the warm up time in the measurements. At a high number of iterations this doesn't make much difference.
I talked to our VM engineers, and they mentioned that the Benchmarks Game does not perform a warm-up before it runs the benchmark. This affects the Dart performance, because the Dart VM does not yet have on-stack replacement (OSR). Both Java and V8 have OSR so they perform better in the Benchmarks Game.
If you look at Dart's benchmark harness, https://github.com/dart-lang/benchmark_harness, you'll see that we perform a small warm up before we start measuring. It would be cool if Benchmarks Game did a warmup, too.
11 comments
[ 4.8 ms ] story [ 31.9 ms ] threadI guess people will have to decide which results are more likely to reflect the performance of their applications. But the extreme nature of the difference is interesting in and of itself.
P.S. I have not looked at the Java code to see if it is idiomatic Java either- And this really make be want to port DeltaBlue to Go...
The Java implementations could also be slightly different.
100,000 iterations takes ~45 seconds to run on my computer.
At a lower number of iterations the DartVM beats the JVM by a significant margin.
The benchmark is easy to run (On linux, maybe macos too). Just check it out, make sure dart, java and gcc are in your path, and execute run.sh.
Note, this code doesn't use the benchmark harness because I wanted to include the warm up time in the measurements. At a high number of iterations this doesn't make much difference.
http://dartogreniyorum.blogspot.com/2013/05/performance-opti...
I talked to our VM engineers, and they mentioned that the Benchmarks Game does not perform a warm-up before it runs the benchmark. This affects the Dart performance, because the Dart VM does not yet have on-stack replacement (OSR). Both Java and V8 have OSR so they perform better in the Benchmarks Game.
If you look at Dart's benchmark harness, https://github.com/dart-lang/benchmark_harness, you'll see that we perform a small warm up before we start measuring. It would be cool if Benchmarks Game did a warmup, too.
We plan to implement OSR but it's not there yet.
This hurts Java too, so at least the use case is apples to apples.
>>It would be cool if Benchmarks Game did a warmup, too.
You should open a ticket/request if you think so?