It's hard to point the finger at any one thing. There have been improvements to the map implementation, the compiler's code generator, the runtime's scheduler and garbage collector, and many parts of the standard library.
At program startup time? Java is a joke in this area.
At memory consumption? Java has a high fixed memory cost. It also uses more memory than Go does. There are fundamental reasons for this. Java needs to keep around runtime type information because the code can change at runtime due to new classes being loaded. Go does not. Java also chose to implement a scheme where any object can serve as a mutex or condition variable. The result is a few words of extra memory per object when compared with Go... usually somewhere in the neighborhood of 8 or 16 bytes per object, depending on architecture. Using memory efficiently is HUGE in modern architectures and Go is better at that.
Go also gives you more control over how your program uses memory. This is because Go has both pointers and value types. The result is that things that would have had to be "boxed" (i.e. separate memory allocations) in Java can be unboxed in Go. If you know C, think of it as the difference between doing a malloc for each structure element, and just having a bunch of structs stored by value inside another struct.
Go is faster at interfacing with native code. Have you checked the speed of JNI lately? Speaking as someone who's written a lot of JNI... it's not pretty.
Java still has the edge in one area: HotSpot has a precise garbage collector, something Go doesn't currently have. But I have a feeling that gap is going to be closed.
Note: I am oversimplifying the unboxing discussion a bit. In theory, it might be possible to unbox certain structures in Java, such as non-nullable final fields. However, Go preserves a lot more of the intentions of the programmer in this regard-- and without knowing how the programmer intended to use the memory, most optimizers will struggle.
For most applications the startup time is irrelevant. The startup time of a server app which runs for months is not an interesting data point. The throughput in requests per second is far more important and this is where the JVM has been very strong (3x faster than Go 1):
Memory consumption is often not a major concern on modern systems, saving 50 MB or even 500 MB is seldom as important as the ability to process more requests per second.
I'm busy learning Go, and am very impressed by the performance gains in 1.1 (and GC improvements). My main concern at the moment is how much less expressive the language is relative to Python, Scala and Ruby. Go is fast, but so is Scala, and Scala is also very expressive compared to Go.
Go has a definite edge in environments where memory footprint is important and simple interfacing with native libraries is critital. Scala and the JVM are still more appealing to me in most other areas.
Not to mention this gives you a lot of flexibility and quite a battle tested backend, whereas the Go equivalents are still mostly in progress and less flexible.
Oh, and in the recent shootout, netty gets to 30,000 rpm, whereas Go peaks around 10,000.
Memory consumption is often not a major concern on modern systems, saving 50 MB or even 500 MB is seldom as important as the ability to process more requests per second.
The difference between 50 MB and 500 MB is that the former is more likely to fit in L3 cache than the latter. Ignore that difference at your peril.
Memory consumption is often not a major concern on modern systems, saving 50 MB or even 500 MB is seldom as important as the ability to process more requests per second.
That's what the original designers of Java thought too. It turns out that this thinking is wrong, though. Main memory may be big, but CPU caches are still small. For example, the laptop I am typing this on has 64 kilobytes of L1 cache, 256 kilobytes of L2 cache, and 3072 kilobytes of L3 cache. That is not a lot. Turns out, when you're spending all your time waiting for main memory, you are not "processing more requests per second." Main memory is at least two orders of magnitude slower than CPU cache.
Java is 2-6 times faster and uses more memory for slightly more code. I'm willing to trade.
Yes, "those are not real world programs etc". In general the shootout gives a quite realistic results (not to mention that anybody who thinks he can do better can go code his own solution). Programs are programs. In my own tests with larger programs Go has been 1/2 the time of Python and slower than Java too. If you're not IO bound, those things matter. If you are IO bound, you could as well run Python, since you aren't missing much.
Startup time doesn't matter at all except if you do scripting or old-style CGI. Java (and Go for that matter) target long running network servers and such.
>Go also gives you more control over how your program uses memory.
On the other hand Java gives most control to the JVM, which is smarter than most programmer's fine tuning and knows a whole lot more about the code's runtime behavior.
The programs in the shootout are nothing like the programs people typically run and write. For instance, most of the programs I write do a lot of I/O. The shootout programs do none.
> Startup time doesn't matter at all
It certainly does in cluster environments where you regularly spin up new server instances. It's also important for command-line tools.
>The programs in the shootout are nothing like the programs people typically run and write. For instance, most of the programs I write do a lot of I/O. The shootout programs do none.
Which is as well, because you only want to benchmark for CPU bound parts of the program.
IO is the same speed for all languages, so it cancels out.
>It certainly does in cluster environments where you regularly spin up new server instances.
Not really a common use case.
>It's also important for command-line tools.
I don't think command-line-tools are Go's or Java's niche.
> IO is the same speed for all languages, so it cancels out.
That's not true, particularly in concurrent systems. Recent improvements to Go's scheduler (that make it more aware of various I/O concerns) have made it 2x faster under some workloads. You don't just optimize the compiler to produce the tightest code, you must also finesse its integration with the rest of the world.
> Not really a common use case.
If you say so.
> I don't think command-line-tools are Go's or Java's niche.
There are a lot of Go command-line tools out there. I think Go is a superb language for this area.
> Why the irony? How many cluster deployments you know
> where you "spin up instances all the time"???
Any cluster environment whose applications abide 12-Factor semantics stands to benefit greatly from low startup times. This describes both SOA environments like Amazon and Netflix, and PaaS environments like Heroku and [what is enabled by] Docker.
> It certainly does in cluster environments where you regularly spin up new server instances. It's also important for command-line tools.
This is certainly true, however not because of the startup time per se which is entirely negligible, especially compared to the time it takes to boot the whole freaking OS, but rather because of the warmup time of the JVM ... for high-volume apps, cold JVMs don't provide a good throughput and so you need a smart load-balancer that knows to not route as many requests as it can to cold machines.
To alleviate that, when the auto-scaling feature kicks in to add new machines on Amazon's EC2 (using their load balancer called ELB), it ads 3 machines instead of one.
Then the auto-scaling feature kicks in again when the latency levels are lower than a threshold for 30 mins, removing machines from the cluster and so it stabilizes itself to the optimal number of machines required.
Compared to Go however, I would still go with Java, because you can do a lot more with a Java server. Our app is both CPU bound and I/O bound and is able to process about 4000 requests per second per server in under 30ms per served request on c1.medium instances. And this is for a real world app that needs to do logging and requests to Memcached/DynamoDB and make predictions in real time.
In my experience, the only way we could have better bang for the buck would be to build the app in C/C++.
It seems odd to me that the language shootout is so "algorithmic", and that there doesn't seem to be a test that works through a lot of string manipulation. Most web apps seem to simply generate a bunch of useless beans and womp through strings in prepared statements and page templates. Benchmark that???
Ideally, yes. But in practice a language and its implementation (whichever one you choose) is a tool. If you're comparing tools, you've got to compare a language with its implementation.
Except that in many cases, e.g. Python, the canonical implementation is not the fastest one.
So of course, the presenter will carefully select the best implementation to prove his arguments, even if there are others available that contradict him/her.
There are fundamental language reasons why Java has a slow startup time. jar files do not contain native bytecode. Instead, they contain standardized, platform-independent bytecode for the Java Virtual machine. What this means in practice is that when you start up, you have to convert all this Java bytecode to native bytecode. This is a CPU and IO-intensive operation. It would be a little like if the first thing your C++ program did was compile itself every time it was run!
There are advantages to the way Java does things. You can distribute a single binary for multiple architectures, for example. However, it does have costs, and one of those is slow startup time.
If you distribute jar files yes, there are however Java compilers that generate native code, if you are willing to pay for them.
Even Sun, nowadays Oracle, sells them as part of the Java SDK for embedded platforms.
That the JVM from Oracle, that most people think as the only way to run Java, does not offer that option out of the box, does not mean other vendors don't do it.
On HN there is usually a pretty high bar for making comments regarding performance, stability, etc...
I'm actually interested in what you have to share regarding performance characteristics of Go versus Java - but a bit more detail than, "Go has a some way to go still and it can be improved in all areas: GC, algorithms, heuristics, code generation," would be useful, particularly in comparison to Java, which you have called out.
Also - I think 36% is a pretty amazing improvement - particularly if Google can persist these performance increases.
I agree that his comment need to provide more data. I also find that HN in general is very quick to downvote comments critical of Go, even well reasoned ones with sources.
In terms of performance data check out quad core results for Go 1 (go 1.1 would need to improve by considerably more than 33% to best the JVM):
The poster's comments regarding GC, heuristics and code generation are vague, but it's no secret that IBM/Sunoracle and co. have very sophisticated GC and JIT implementations which they have been optimizing for 15+ years and it shows through in the benchmark results.
I like Go, I just think that we need to be wary of expecting it to be as fast as the JVM and even MSCLR straight out the gate. It will likely take years to reach that level of sophistication.
If anything, I think that Go has received a disproportionate amount of criticism and skepticism because it was developed at Google. Go is very mature these days - way more mature than most languages ever get - and that's the result of hard work by the contributors, not its pedigree.
I've seen posts where "it's from Google" is cited as sufficient explanation for taking an interest in the language. If Go was a random project on GitHub there is very little chance that I would have given it a second look, let alone learning it (which I am presently doing). I have great respect for projects which start out as random GitHub projects and succeed without having the advantage of a name like Google to give them street cred.
Most of the negative Go posts I've seen are to do with limitations of the language and performance relative to JVM languages. As much as I think go has a lot going for it I'm not about to claim that it's anywhere near as expressive as Python or even Scala. There has also been some frustration about denials over the performance gap between Go and the JVM despite very clear data in favor of the JVM. I'm not as bothered by the performance gap as I am about the head-in-the-sand denials. The JVM's have been optimized for 15+ years, so it's ok to admit it may be a few more years before performance can match the JVM.
> I've seen posts where "it's from Google" is cited as sufficient explanation for taking an interest
That might be enough to draw interest, but what sustains that interest is the quality of the work and the utility of the environment.
> If Go was a random project on GitHub
Where it's hosted is irrelevant. It's about the work. Plenty of active, popular projects are hosted on GitHub. Some of them were just "random projects on GitHub" when they started. What distinguishes them is that there are motivated groups of people working on them.
Good thing one of the key contributors to the HotSpot JIT is part of the core Go team (please don't ask me to spell the Swiss guy's name). Give them another couple years, Go may well pull up to the same "asymptote" for GC performance, tuning, and such.
Good point. It turns out that the quad core benchmarks on 32bit which I linked to are already using Go 1.1beta, so any speed gains for Go 1.1 are already factored in:
Notice on the list provided by `trailfox` the "≈ CPU Load" column -- when the programs are written to use 4 cores "they're" allowing Go to access 4 cores.
That said, Java still beats Go quite readily in this benchmark spread, particularly in regex and binary-trees where Go does extremely poor. It should be noted that lots of languages implement their regex libraries in C for speed, while Go's implementation is written in Go. Still, I think it should do better than this.
That Java beats Go overall is not particularly surprising, Java is very mature and it's jit-compiler has very advanced optimizations, Perhaps by Go 2.0 we'll see where Go 'lands' in terms of performance, there's obviously alot more to be had.
In addition to the improvements from the improved scheduler, use of accept4 on Linux, better GC and code generation, Brad Fitzpatrick has been giving a lot of love to the "net/http" package. Here is a small example: https://plus.google.com/u/0/115863474911002159675/posts/L3o9...
It's a pity this is a comparison with Go 1.0.3. That's so old as to be a almost useless comparison given that everyone who's serious about Go uses something from the hg repository.
It's not very common. John (the GP) uses the Go bleeding edge at CloudFlare because he needs some of the recent improvements. To do this, he really needs to know what he's doing (and he does). Typical Go users just use the stable branch.
Yeah, especially now that the tip is Go 1.1 post-beta.
It's pretty common in any language that's seeing significant development, targeting a version that's not going to be too long in the tooth by the time you want to go to production.
1.0.3 was the last stable release. 1.1 is the next stable release. Not everyone feels confident running testing repos for production systems (even if they are reported to be stable)
88 comments
[ 3.0 ms ] story [ 71.6 ms ] threadGo has a some way to go still and it can be improved in all areas: GC, algorithms, heuristics, code generation, ...
At program startup time? Java is a joke in this area.
At memory consumption? Java has a high fixed memory cost. It also uses more memory than Go does. There are fundamental reasons for this. Java needs to keep around runtime type information because the code can change at runtime due to new classes being loaded. Go does not. Java also chose to implement a scheme where any object can serve as a mutex or condition variable. The result is a few words of extra memory per object when compared with Go... usually somewhere in the neighborhood of 8 or 16 bytes per object, depending on architecture. Using memory efficiently is HUGE in modern architectures and Go is better at that.
Go also gives you more control over how your program uses memory. This is because Go has both pointers and value types. The result is that things that would have had to be "boxed" (i.e. separate memory allocations) in Java can be unboxed in Go. If you know C, think of it as the difference between doing a malloc for each structure element, and just having a bunch of structs stored by value inside another struct.
Go is faster at interfacing with native code. Have you checked the speed of JNI lately? Speaking as someone who's written a lot of JNI... it's not pretty.
Java still has the edge in one area: HotSpot has a precise garbage collector, something Go doesn't currently have. But I have a feeling that gap is going to be closed.
Note: I am oversimplifying the unboxing discussion a bit. In theory, it might be possible to unbox certain structures in Java, such as non-nullable final fields. However, Go preserves a lot more of the intentions of the programmer in this regard-- and without knowing how the programmer intended to use the memory, most optimizers will struggle.
http://www.techempower.com/blog/2013/03/28/framework-benchma...
Memory consumption is often not a major concern on modern systems, saving 50 MB or even 500 MB is seldom as important as the ability to process more requests per second.
I'm busy learning Go, and am very impressed by the performance gains in 1.1 (and GC improvements). My main concern at the moment is how much less expressive the language is relative to Python, Scala and Ruby. Go is fast, but so is Scala, and Scala is also very expressive compared to Go.
Go has a definite edge in environments where memory footprint is important and simple interfacing with native libraries is critital. Scala and the JVM are still more appealing to me in most other areas.
rather than this: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
(not to mention the 2 other Java files to setup the server).
Oh, and in the recent shootout, netty gets to 30,000 rpm, whereas Go peaks around 10,000.
Based on these results Go and Netty should be much closer: https://gist.github.com/errnoh/5320784
Here's Go 1.1 vs 1.0.3 on a bit beefier machine: https://gist.github.com/errnoh/5320784
Next benchmark will have Go 1.1 running, can't promise that it'll scale as well as on this server but it'll be significant change anyways.
And most of all, main part of the language is that it's intuitive to write and comes with a clear and readable spec and standard library.
edit: updated gist with Netty using the same configuration
The difference between 50 MB and 500 MB is that the former is more likely to fit in L3 cache than the latter. Ignore that difference at your peril.
Tell that to the folks that have to pay for cloud or VPS servers... Also, less memory usage means higher CPU cache hit rates and fewer page faults.
are you _really_ sure it is not just your bias that is preventing you from "expressing" what you want? ie. you want to write python/java in Go?
http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...
That's what the original designers of Java thought too. It turns out that this thinking is wrong, though. Main memory may be big, but CPU caches are still small. For example, the laptop I am typing this on has 64 kilobytes of L1 cache, 256 kilobytes of L2 cache, and 3072 kilobytes of L3 cache. That is not a lot. Turns out, when you're spending all your time waiting for main memory, you are not "processing more requests per second." Main memory is at least two orders of magnitude slower than CPU cache.
No. In actual performance for most cases of algorithmic code.
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
Java is 2-6 times faster and uses more memory for slightly more code. I'm willing to trade.
Yes, "those are not real world programs etc". In general the shootout gives a quite realistic results (not to mention that anybody who thinks he can do better can go code his own solution). Programs are programs. In my own tests with larger programs Go has been 1/2 the time of Python and slower than Java too. If you're not IO bound, those things matter. If you are IO bound, you could as well run Python, since you aren't missing much.
Startup time doesn't matter at all except if you do scripting or old-style CGI. Java (and Go for that matter) target long running network servers and such.
>Go also gives you more control over how your program uses memory.
On the other hand Java gives most control to the JVM, which is smarter than most programmer's fine tuning and knows a whole lot more about the code's runtime behavior.
The programs in the shootout are nothing like the programs people typically run and write. For instance, most of the programs I write do a lot of I/O. The shootout programs do none.
> Startup time doesn't matter at all
It certainly does in cluster environments where you regularly spin up new server instances. It's also important for command-line tools.
Which is as well, because you only want to benchmark for CPU bound parts of the program.
IO is the same speed for all languages, so it cancels out.
>It certainly does in cluster environments where you regularly spin up new server instances.
Not really a common use case.
>It's also important for command-line tools.
I don't think command-line-tools are Go's or Java's niche.
That's not true, particularly in concurrent systems. Recent improvements to Go's scheduler (that make it more aware of various I/O concerns) have made it 2x faster under some workloads. You don't just optimize the compiler to produce the tightest code, you must also finesse its integration with the rest of the world.
> Not really a common use case.
If you say so.
> I don't think command-line-tools are Go's or Java's niche.
There are a lot of Go command-line tools out there. I think Go is a superb language for this area.
Why the irony? How many cluster deployments you know where you "spin up instances all the time"???
Compared to general server / network application programming done in both Java and Go it's LESS than a drop in the bucket.
If you work somewhere that this does not hold, you could also be an outlier.
http://12factor.net
This is certainly true, however not because of the startup time per se which is entirely negligible, especially compared to the time it takes to boot the whole freaking OS, but rather because of the warmup time of the JVM ... for high-volume apps, cold JVMs don't provide a good throughput and so you need a smart load-balancer that knows to not route as many requests as it can to cold machines.
To alleviate that, when the auto-scaling feature kicks in to add new machines on Amazon's EC2 (using their load balancer called ELB), it ads 3 machines instead of one.
Then the auto-scaling feature kicks in again when the latency levels are lower than a threshold for 30 mins, removing machines from the cluster and so it stabilizes itself to the optimal number of machines required.
Compared to Go however, I would still go with Java, because you can do a lot more with a Java server. Our app is both CPU bound and I/O bound and is able to process about 4000 requests per second per server in under 30ms per served request on c1.medium instances. And this is for a real world app that needs to do logging and requests to Memcached/DynamoDB and make predictions in real time.
In my experience, the only way we could have better bang for the buck would be to build the app in C/C++.
How do you know what programs people typically run and write?
Assumption? Guess? Data?
But you can draw the same conclusion by browsing GitHub.
1) I think the way in which the benchmarks game programs are most unlike a lot of other programs is that these are tiny tiny toy programs.
"Toy programs" as-in -- http://books.google.com/books?id=gQ-fSqbLfFoC&lpg=PP1...
And that's intentional.
2) Why do you say "...I/O. The shootout programs do none."?
regex-dna, k-nucleotide, and reverse-complement do read data in FASTA format before processing.
(fasta and reverse-complement also write data in FASTA format -- but redirected to /dev/null)
Not TBs of I/O but not none.
It is always a matter if you are using a JVM (from which vendor) or compiling to native code.
Language != Implementation
Ideally, yes. But in practice a language and its implementation (whichever one you choose) is a tool. If you're comparing tools, you've got to compare a language with its implementation.
Which from 6g, gogcc, go express you pick for Go?
Which from OpenJDK, Oracle JVM, IBM J9, gcj, RoboVM, Aonix Perc ... you pick for Java?
Which from Sun Forte, IBM xlc, hpcc, gcc, clang, CINT, Visual C++, ... you pick for C or C++?
Which from CPython, IronPython, PyPy, JPython, ... you pick for Python?
For all others there is a definitive, used in the overwhelming majority of cases, implementation:
Go: 6g Java: Oracle JVM Python: CPython Ruby: Matz's Haskell: GHC ...
So of course, the presenter will carefully select the best implementation to prove his arguments, even if there are others available that contradict him/her.
There are advantages to the way Java does things. You can distribute a single binary for multiple architectures, for example. However, it does have costs, and one of those is slow startup time.
Even Sun, nowadays Oracle, sells them as part of the Java SDK for embedded platforms.
That the JVM from Oracle, that most people think as the only way to run Java, does not offer that option out of the box, does not mean other vendors don't do it.
I'm actually interested in what you have to share regarding performance characteristics of Go versus Java - but a bit more detail than, "Go has a some way to go still and it can be improved in all areas: GC, algorithms, heuristics, code generation," would be useful, particularly in comparison to Java, which you have called out.
Also - I think 36% is a pretty amazing improvement - particularly if Google can persist these performance increases.
In terms of performance data check out quad core results for Go 1 (go 1.1 would need to improve by considerably more than 33% to best the JVM):
http://benchmarksgame.alioth.debian.org/u32q/benchmark.php?t...
Also see Netty (Java) outperforming go by a wide margin here: http://www.techempower.com/blog/2013/03/28/framework-benchma...
The poster's comments regarding GC, heuristics and code generation are vague, but it's no secret that IBM/Sunoracle and co. have very sophisticated GC and JIT implementations which they have been optimizing for 15+ years and it shows through in the benchmark results.
I like Go, I just think that we need to be wary of expecting it to be as fast as the JVM and even MSCLR straight out the gate. It will likely take years to reach that level of sophistication.
I am sure if Go wasn't being done at Google, most hackers would actually ignore it.
Just check how successful the Go like predecessors from the authors were, when working at other companies.
I think the language suffers from an hallo effect.
Makes it easier to decide whether Go is the right language choice for the problem at hand. Quite often it won't be and that's fine.
Most of the negative Go posts I've seen are to do with limitations of the language and performance relative to JVM languages. As much as I think go has a lot going for it I'm not about to claim that it's anywhere near as expressive as Python or even Scala. There has also been some frustration about denials over the performance gap between Go and the JVM despite very clear data in favor of the JVM. I'm not as bothered by the performance gap as I am about the head-in-the-sand denials. The JVM's have been optimized for 15+ years, so it's ok to admit it may be a few more years before performance can match the JVM.
That might be enough to draw interest, but what sustains that interest is the quality of the work and the utility of the environment.
> If Go was a random project on GitHub
Where it's hosted is irrelevant. It's about the work. Plenty of active, popular projects are hosted on GitHub. Some of them were just "random projects on GitHub" when they started. What distinguishes them is that there are motivated groups of people working on them.
That is pretty much the zinger. If Google hadn't been involved, Go probably wouldn't be were it is at all.
I would hope that we hackers would pay at least some attention to any project involving Ken Thompson and Rob Pike though.
http://research.google.com/pubs/author96.html
http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
http://benchmarksgame.alioth.debian.org/u32q/benchmark.php?t...
The 64 bit quad core benchmark is using go 1.0.3:
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
Note: The much faster version Go's DNA regex that uses PCRE rather than "regexp", but it is currently broken due to a build error. (http://benchmarksgame.alioth.debian.org/u64/program.php?test...)
Go uses less memory than Java 7, but then again so does Java 1.1
http://benchmarksgame.alioth.debian.org/u32q/program.php?tes...
Note that:
1) It only ever uses one core, because
2) It isn't configured with GOMAXPROCS, which therefore defaults to 1.
So, sure, they have an N-core machine, but they're only allowing go to access 1 core.
Notice on the list provided by `trailfox` the "≈ CPU Load" column -- when the programs are written to use 4 cores "they're" allowing Go to access 4 cores.
The go program in question was written without that instruction; therefore, it's only allowed to use one core.
Only "allowed" by who?
If you mean that "the Go program in question" is by-design a sequential program then -- yes of course it is!
The contributed Go programs that were written for multi-core use more than one core -- http://benchmarksgame.alioth.debian.org/u32q/program.php?tes...
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
That said, Java still beats Go quite readily in this benchmark spread, particularly in regex and binary-trees where Go does extremely poor. It should be noted that lots of languages implement their regex libraries in C for speed, while Go's implementation is written in Go. Still, I think it should do better than this.
That Java beats Go overall is not particularly surprising, Java is very mature and it's jit-compiler has very advanced optimizations, Perhaps by Go 2.0 we'll see where Go 'lands' in terms of performance, there's obviously alot more to be had.
http://benchmarksgame.alioth.debian.org/u32q/benchmark.php?t...
It's pretty common in any language that's seeing significant development, targeting a version that's not going to be too long in the tooth by the time you want to go to production.