On average, compiled languages
consumed 120J to execute the solutions, while for
virtual machine and interpreted languages this value was
576J and 2365J, respectively.
On a broad scale, yes. But it also proves that the efficiency of particular languages depends on the problem at hand. For example, if your problem is searching for a pattern in large strings, a simple Perl program using the builtin regex support will easily beat a C/C++ program that hand-rolls pattern matching, because Perl's regex engine has been optimized for over two decades.
On the topic of mobile devices, I recall the Nokia N900, which ran mostly C and C++ software. Its UI performance was comparable to contemporary smartphones even though it only had 256 MB RAM and a single-core 600 MHz CPU. (Of course, a major part of today's phones' slow UI is all the bullshit and spying that apps are doing in the background.)
Most Android apps use a handful of native code libraries which are bundled inside the apk. It's usual nowadays to only write the UI using Java and use C for the rest.
The summary results for Swift do not reflect the individual tests. There is a single anomalous test, reflex-redux, where the Swift implementation sucks badly.
Anecdotally, Swift string handling makes it easy to do the right thing when handling Unicode strings, but also makes it easy to burn up several orders of magnitude too much CPU when you use those functions for simple tasks. I once saved three orders of magnitude clock time by dropping to POSIX IO and C run time library string handling for an application that needed to read a text file containing 10000 lines of 81 digits at launch and process them as single digits.
Before all of you go out taking these results as definite truth, remember that a language benchmark is only as good as the particular implementations. A $LANGUAGE fanboy can easily game the benchmark by overoptimizing the $LANGUAGE implementations into unmaintainable atrocities that perform extremely well, thus making people think that average $LANGUAGE programs will exhibit the same performance/efficiency/resource usage characteristics.
There's no easy way to guard against this problem when you're only a handful of researchers. If you have a large team with a background in a lot of languages, you can have them write and peer-review implementations that are idiomatic (i.e. representative of the language's practical usage).
And before you all go out taking the possibility of scepticism as an actual motivation to be sceptical, remember that conclusions do not need to be proven beyond all doubt to be reasonably believed.
I've always suspected using "like really productive" languages creates it's own scalability problem in production. Ruby/Python/Node/PHP consuming anywhere from 2x to 15x the amount of energy and time as a language like Java/C# comes as no surprise, which likely means you'll need a corresponding increase in servers for particular use cases.
What's not a surprise is C stomping everything with a lead boot. I am surprised that Fortran is a lower than it is (considering its role in HPC/research). It's no surprise to see Java leading the virtual machine pack on speed, I thought it'd take a larger hit on energy consumption due to JIT.
But Rust, WOW. Have we finally created the perfect language? Always something I've wanted to learn, now I think I have a good reason!
I don't intuitively understand why Go was so space efficient (#2 and ahead of C) but then so time+energy inefficient wrt C (3.23 and 2.83 vs 1.00 and 1.00).
22 comments
[ 3.1 ms ] story [ 63.7 ms ] threadOn average, compiled languages consumed 120J to execute the solutions, while for virtual machine and interpreted languages this value was 576J and 2365J, respectively.
On the topic of mobile devices, I recall the Nokia N900, which ran mostly C and C++ software. Its UI performance was comparable to contemporary smartphones even though it only had 256 MB RAM and a single-core 600 MHz CPU. (Of course, a major part of today's phones' slow UI is all the bullshit and spying that apps are doing in the background.)
But would be nice to have some mobile apps written with C.
Since it's a single hybrid language, it's trivial to remove slower features fro, performance-intensive parts.
See my UIKonf talk https://www.youtube.com/watch?v=kHG_zw7%205SjE&feature=youtu...
Or my book: https://www.amazon.com/gp/product/0321842847/ref=as_li_tl?ie...
-- Edit --
My bad, I only looked at the paper. The results (https://sites.google.com/view/energy-efficiency-languages/re...) show that Swift and Go are doing about as well as Java. (Which means they're right behind C, C++, Rust and Fortran).
Anecdotally, Swift string handling makes it easy to do the right thing when handling Unicode strings, but also makes it easy to burn up several orders of magnitude too much CPU when you use those functions for simple tasks. I once saved three orders of magnitude clock time by dropping to POSIX IO and C run time library string handling for an application that needed to read a text file containing 10000 lines of 81 digits at launch and process them as single digits.
There's no easy way to guard against this problem when you're only a handful of researchers. If you have a large team with a background in a lot of languages, you can have them write and peer-review implementations that are idiomatic (i.e. representative of the language's practical usage).
What's not a surprise is C stomping everything with a lead boot. I am surprised that Fortran is a lower than it is (considering its role in HPC/research). It's no surprise to see Java leading the virtual machine pack on speed, I thought it'd take a larger hit on energy consumption due to JIT.
But Rust, WOW. Have we finally created the perfect language? Always something I've wanted to learn, now I think I have a good reason!
Any clues?