time ./drip -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 'a=1;puts a'
1
./drip -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 0.03s user 0.03s system 4% cpu 1.282 total
time java -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 'a=1;puts a'
1
java -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 3.65s user 0.12s system 183% cpu 2.056 total
Interesting!! if it runs rails effectively, this could be awesome for jruby.
This seems like a really clever solution. I remember trying to set up nailgun a while ago back when I was experimenting with Clojure because the startup time for running a program was so long, but I could never quite get it to work. I missed being able to run python myprog.py and getting instant feedback.
AOT already means something to clojurists. It means removing the extra runtime compilation phase of .clj into classfiles. The JIT is what the JVM does, which still occurs just like before.
So obvious, trade-off space for time, yet I wouldn't have thought of it... I mean, I've thought about this problem, written a persistent JVM solution, and didn't think of it. Memory is cheaper than my intuition realises.
I wonder how many other "obvious" solutions I'm missing like this?
EDIT for the code I tried, user time is almost 3 times faster, but real time is only around 10% better... I don't understand linux well enough to know why - anyone care to explain please? EDIT Yes, drip had already run. (I picked typical times from about 10 runs each).
$ time java...
real 0m1.466s
user 0m1.216s
sys 0m0.180s
$ time drip...
real 0m1.378s
user 0m0.412s
sys 0m0.260s
BTW: For server-like workloads, an advantage of a persistent JVM is that it gets dramatically faster over repeated runs of the same code, as it improves its hotspot-style adaptive optimisations.
I really like his quickstart "standalone" installation.
Did you do two exact same drip runs in a row? It has to hash and preload a JVM with the same classpath setup to work.
$ time java -jar clojure-1.3.0.jar add_1_and_1.clj
real 0m0.690s
user 0m0.827s
sys 0m0.030s
$ time drip -jar clojure-1.3.0.jar add_1_and_1.clj
real 0m0.879s
user 0m1.020s
sys 0m0.040s
$ time drip -jar clojure-1.3.0.jar add_1_and_1.clj
real 0m0.177s
user 0m0.033s
sys 0m0.020s
$ time drip -jar clojure-1.3.0.jar add_1_and_1.clj
real 0m0.176s
user 0m0.050s
sys 0m0.017s
Hasn't anyone ported zygote over to desktop Linux/Windows? You just keep the preconfigured jvm process running and fork it indefinitely for each new process. You'd still suffer some overhead depending on each application but that's just expected anyway. The jvm startup overhead isn't.
I don't know if such a port exists, but I do know that I wouldn't want one. Dalvik's performance characteristics are scary-bad and there are plenty of incompatibilities with Java libraries that do any sort of weaving or code generation (so you still can't use Groovy, Rhino, etc.).
... and there are plenty of incompatibilities with Java libraries that do any sort of weaving or code generation ...
Ah, good to know. I've mostly just had to bring in things like a database library, and in one case LuaJ. None of those did need to do bytecode generation.
Yeah, for that you'll be fine, but take Rhino for example - Rhino offers, through LiveConnect, the ability to create JavaScript objects, at runtime, that satisfy Java interfaces. Which is cool, and useful for a lot of stuff. But the version of Rhino you can use on Dalvik has all of the bytecode generation disabled (obviously) and as a consequence, it can't emit the necessary bytecode to create the interface-fulfilling Java wrapper to make the JS object do what you want. It's generally bogus.
Also it's really hard to express how much slower Dalvik is than HotSpot in the general case, too. It seems faster with the Android 4.2 version (by about 30%, in my highly unofficial testing) but it's still pretty agonizingly slow.
Interesting idea (especially, hashing based on command line options, and probably also the libraries in the classpath); I should add this to Clove (http://hovel.ca/clove : a small persistent-jvm that I wrote for *nix, with VERY fast connection time, e.g. suitable for scripts; sorry for shameless plug).
They do slightly different things. I think each can benefit by incorporating the main idea of the other:
- Working on multiple projects who need different JVMs running at the same time can be a hassle in clove (you have to create a service for each), so it can be fixed by doing that the same way drip does.
- Running scripts and having a persistent JVM is still very useful, and clove (albeit I haven't cleaned up its code properly) gives a terminal into the JVM, really fast. So, drip can use the same technique (passing capabilities; a *nix feature) to let you hook into a JVM very quickly. [see the "screenshot" part at the bottom of http://hovel.ca/clove]
My experiences with drip were very mixed (on Ubuntu 10.04), it did not make a stable, reliable impression on me. I wouldn't recommend it for professional use.
A github issue (#34) was opened, and you released a changed version soon after, but it remained unusable for me (in the meantime I upgraded to 12.04 and did not test drip since then).
What does this do exactly? Will it permanently change lein to use drip? Or only temporarily for whichever command replaced the "...". Also, do you know if the advice here: https://github.com/flatland/drip/wiki/Clojure
works with lein2?
I read his explanation but I don't have a great understanding of why a JVM would need to clean up like this. Doesn't it have a Garbage collector just for that? Why does it get slower over time? The only thing I know that gets slower over time without you doing anything special is my dad's mac os x word-only macbook pro.
As a further optimization, why not memcpy an uncorrupted JVM to some backup place in memory and then when you want to 'reboot' just memcpy the image back again?
> It keeps a fresh JVM spun up in reserve with the correct classpath and other JVM options so you can quickly connect and use it when needed
So I assume it doesn't help if you are launching JVMs very rapidly (like, scripting stuff in a tight loop). Slow JVM launching has pretty much killed languages such as Groovy for scripting for me, because once I start using them in loops things get horribly slow.
It's not like the stuff inside the script can't be standalone classes of the "do one thing and do it well" variety --they just don't get to be processes.
On the other hand, you may not want to reimplement sed, grep, etc in your scripting language of choice. There are times where a good "perl -e" can be very useful in a shell script.
46 comments
[ 4.1 ms ] story [ 71.4 ms ] threadtime java -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 'a=1;puts a' 1 java -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 3.65s user 0.12s system 183% cpu 2.056 total
Interesting!! if it runs rails effectively, this could be awesome for jruby.
I recently moved one of my project to jruby and the startup time is the only thing I'm complaining about.
I prefer swank over nailgun, even with Vim.
I meant AOT to native code.
Although OpenJDK/Oracle JVM don't offer this option, other vendors do.
EDIT: Hmm, I encountered this: "Could not connect to compilation daemon after 300 attempts." but re-running it ("scala") works.
I wonder how many other "obvious" solutions I'm missing like this?
EDIT for the code I tried, user time is almost 3 times faster, but real time is only around 10% better... I don't understand linux well enough to know why - anyone care to explain please? EDIT Yes, drip had already run. (I picked typical times from about 10 runs each).
BTW: For server-like workloads, an advantage of a persistent JVM is that it gets dramatically faster over repeated runs of the same code, as it improves its hotspot-style adaptive optimisations.I really like his quickstart "standalone" installation.
WARNING "drip kill" crashed my system. The kill functions are kill_jvms and kill_jvm (https://github.com/flatland/drip/blob/develop/bin/drip). I'm using an older ubuntu 10.04 LTS.
You'd just need to convert all your .jars and that's it. I haven't heard of any other incompatibilities with standard Java.
Edit: Is there a port of Dalvik to amd64? A cursory search did not find one.
Ah, good to know. I've mostly just had to bring in things like a database library, and in one case LuaJ. None of those did need to do bytecode generation.
Also it's really hard to express how much slower Dalvik is than HotSpot in the general case, too. It seems faster with the Android 4.2 version (by about 30%, in my highly unofficial testing) but it's still pretty agonizingly slow.
- Working on multiple projects who need different JVMs running at the same time can be a hassle in clove (you have to create a service for each), so it can be fixed by doing that the same way drip does.
- Running scripts and having a persistent JVM is still very useful, and clove (albeit I haven't cleaned up its code properly) gives a terminal into the JVM, really fast. So, drip can use the same technique (passing capabilities; a *nix feature) to let you hook into a JVM very quickly. [see the "screenshot" part at the bottom of http://hovel.ca/clove]
https://github.com/flatland/drip/issues/36
Here's my workaround to use drip to start up lein repls instantly:
https://gist.github.com/4103370
Should work with lein1 & lein2.
So I assume it doesn't help if you are launching JVMs very rapidly (like, scripting stuff in a tight loop). Slow JVM launching has pretty much killed languages such as Groovy for scripting for me, because once I start using them in loops things get horribly slow.
It's not like the stuff inside the script can't be standalone classes of the "do one thing and do it well" variety --they just don't get to be processes.
Does anyone know what kind of errors the author is referring to? Does any long running JVM instance run into these issues?