iOS does not allow you to run any sort of non-apple just-in-time compiler (there is no public API to get executable mmaped pages to be precise), so a no go, you cannot.
It's more than that: nothing outside of the built-in applications is allowed to allocated executable memory at all. You can't even use JSC's JIT (i.e., that of Apple's JS engine) in third-party applications.
> You can't even use JSC's JIT (i.e., that of Apple's JS engine) in third-party applications.
Although this very specific one may get fixed through remote views, which apple is currently toying with (a webview would live in its own "webview process" and thus embedded webview could finally get native JIT-ed speeds)
Regarding non-synthetic benchmarks, quite recently I've tested pypy as a "backend" to Gentoo's package manager (portage). System was quite not up-to-date, lots of packages and dependencies, this is how much time did it take portage to figure out what to update:
python2.7: 323.13s user 6.50s system 99% cpu 5:30.59 total
python3.2: 271.40s user 6.33s system 99% cpu 4:38.64 total
pypy1.9: 168.28s user 5.95s system 99% cpu 2:55.16 total
There is very limited data for non-artificial and non-numeric benchmarks for Lua and Python, so treat it all with a grain of salt.
PyPy mostly cares about large scale programs (thanks to a large quantity of Python libraries it's not that hard), while LuaJIT usually cares for smaller stuff.
Last time I tried to run richards (which is the largest example I found for both Lua and Python), PyPy was able to eventually outperform LuaJIT slightly, given enough iterations, however LuaJIT warmup time was very very impressive and it was much faster at the beginning.
From a very unscientific perspective I would expect LuaJIT to mostly outperform PyPy, especially on stuff where it's feasible to compare, like by implementing small numeric kernels in both Lua and Python, but I haven't measured a whole lot in this area. LuaJIT definitely outperforms PyPy on the computer language shootout benchmarks. I would also expect the field to be more leveled if you implement say a webserver in both, caring about performance, but this is really on the guessing side.
You can also download the full experimental suite and check out the particular benchmark(s) yourself. In my opinion, fijal has provided an accurate summary of the situation as we currently understand it.
The problem really is Python import semantics. As soon as you can provide me with a way to say "this imported code is identical to the one you loaded before", I'm fine with providing a way to save the JIT state. Note that "identical" means also module globals, state of sys.modules etc. etc. In short, the assumptions are all the same.
> As soon as you can provide me with a way to say "this imported code is identical to the one you loaded before", I'm fine with providing a way to save the JIT state.
That would not be sufficient. Python modules can execute arbitrary code at import time, so the behaviour could depend on a number of external factors, such as environment variables, other files, current time, response from a remote service (?!), etc...
That would not be sufficient in some corner cases. I think for a competent group running web application servers, which knows it needs to control such external factors, it could well be enough.
but this is not the attitude you have to have when writing compilers. from the compiler standpoint, it has to be correct, even in corner cases (which are not even as corner as you would like them to be).
A competent group can leave a dormant process that communicates over pipe for example and then the warmup issue disappears (it does not really appear for long running web application servers to start with)
Have you given up on automatic handling being feasible? It seems like the performance wins could be enough to justify something like a module-level constant explicitly stating that you're not doing metaprogramming, monkey-patching, etc. and allowing implementors to optimize as desired.
The Lua port of Richards I've seen is naive and nobody within his right mind would write Lua code like that. As has been pointed out elsewhere, Richards is a very atypical benchmark and different implementation styles have very different performance characteristics.
In fact, PyPy wins ONLY on Richards by 1.2x and ONLY for a specific iteration count. But LuaJIT wins substantially on ALL OTHER BENCHMARKS where something is actually compiled (i.e. not regex stuff). Your paper clearly shows that in the appendix, but no matter what, you're still pointing out that one little result.
And, even worse, you're trying to draw conclusions from this outlier. I have absolutely no idea how you could possibly come to the conclusion that this makes PyPy more suitable for 'large scale' programs? And that LuaJIT is not? That's not just unscientific -- it's dishonest and disparaging.
Also, constantly repeating that 'hand-crafted JIT compilers' are somehow inferior to PyPy's overly complex approach is getting really old by now. LuaJIT runs on more architectures and more platforms than PyPy ever will. And it's vastly ahead in terms of optimizations. Now, please compare the timeframe all of this was implemented and the size of our teams. Ok, so what was the better deal, again?
And, no, the semantics of Lua and Python are not that far apart as you'd like everyone to believe. Dispatch is more complex in Python, but that can be taken care of in a central place in the code. Most library code can be written in terms of lower-level code. And so on. There are simple solutions for all of that. And none of that warrants a clumsy meta-tracing approach.
I know myself that I had a high regard for LuaJIT before benchmarking, and an even higher regard for it afterwards. Richards is an oldish benchmark, and none of the subsequent versions (Lua, Python, Converge) is exactly "native", so I'm not sure that any of those languages has an advantage. It's a datapoint, no more, no less. As to which VM would be faster on "large" programs - well, personally I agree with your implication that it's a matter of conjecture, and we'll probably never know because it's hard to write large cross-language benchmarks.
I do, however, think you might be a little harsh on meta-tracing. It's an interesting approach, in no small part because it allows VMs to be written for languages other than Python. LuaJIT can be retargeted too (I've been impressed by Thomas Schilling's work), but I think it would be fair to say that it's harder work than writing a new RPython interpreter. In other words, IMHO, there is plenty of scope for both LuaJIT and RPython/PyPy to co-exist happily alongside each other.
Seriously, I would not try to say LuaJIT is worse than PyPy, ever. I picked richards, because it's the best benchmark from the set I know, while still being not a very good one. I'm all for saying "this is a bad benchmark".
I think you're putting words into my mouth. I did not say PyPy is better for large scale programs, I said we have more focus and I derive it from what we use as PyPy benchmarks. Since it's impossible to compare, because we lack programs we don't actually know, but my gut feeling would be that LuaJIT would be slightly faster on something large.
The problem with Python (and why we choose our approach instead of hand crafting the JIT) is that Python is a very large language, with lots of builtin modules and types interfering in very unobvious ways, while Lua is not. This is why we decided it makes more sense to go that way, we probably wouldn't do it for Lua. People tried to write a JIT by hand for Python and failed so far, I'm however fine with the explanation we're just too dumb to do that.
Overall I find your answer way too agressive - I didn't claim PyPy is better, I mostly pointed out to things I know and added it's very unscientific and that scientific approach is very hard to get. Also your attack on our meta-tracing is unfounded - Python is just a very very large language with strange corner cases and we have to somehow deal with it. Feel free to implement it differently, but I'm happy with our approach.
You did say that LuaJIT "usually cares for smaller stuff," implying that it isn't optimal (or as optimal as PyPy) for larger stuff. I could see how someone who had worked for years to optimize LuaJIT for all cases could find that disparaging. It sounds like you're writing off LuaJIT as only meant for toy programs.
Well. the luaJIT benchmark page does not contain any large benchmarks, while it does contain computer language shootout ones which are small. There might be many many legitimate reasons for that, but it does imply that LuaJIT cares for small stuff more, since all the public benchmarks are small.
The HTM buffer is too small for any language for doing what I'm talking about, which is long transactions. Haswell uses its L1 Data cache as buffer --- which means at most 32KB or 64KB, and only if you're on a lucky day. (Actually it depends on cache collision and alignment issues: it can in theory overflow with only 9 memory accesses, but I'd expect the numbers to be generally more than half the full cache size.)
The HTM buffer is big enough if the goal is "only" to remove the GIL. It's going to be a 10-lines patch to CPython or PyPy (yes, really!). The GIL is often made into a big issue, but in my opinion removing it isn't the final solution to all multicore troubles in Python: that's just the first step.
35 comments
[ 3.9 ms ] story [ 70.6 ms ] threadWhat's the memory usage like? That used to be a problem for anyone considering PyPy on mobile chipsets.
Although this very specific one may get fixed through remote views, which apple is currently toying with (a webview would live in its own "webview process" and thus embedded webview could finally get native JIT-ed speeds)
EDIT: back for now
I ask since they have similar goals for their respective languages.
[1] http://luajit.org
PyPy mostly cares about large scale programs (thanks to a large quantity of Python libraries it's not that hard), while LuaJIT usually cares for smaller stuff.
Last time I tried to run richards (which is the largest example I found for both Lua and Python), PyPy was able to eventually outperform LuaJIT slightly, given enough iterations, however LuaJIT warmup time was very very impressive and it was much faster at the beginning.
From a very unscientific perspective I would expect LuaJIT to mostly outperform PyPy, especially on stuff where it's feasible to compare, like by implementing small numeric kernels in both Lua and Python, but I haven't measured a whole lot in this area. LuaJIT definitely outperforms PyPy on the computer language shootout benchmarks. I would also expect the field to be more leveled if you implement say a webserver in both, caring about performance, but this is really on the guessing side.
Is this benchmark data-intensive? One known weak point of LuaJIT is its GC, which is slated for a rewrite: http://wiki.luajit.org/New-Garbage-Collector
If "richards" allocates lots of objects that accumulate and need to be GC'd, this could explain why PyPy wins eventually.
http://labs.oracle.com/people/mario/java_benchmarking/richar...
I'd be very surprised if PyPy beat LuaJIT for anything that was numerical or otherwise had a fixed working set.
A draft paper where C F Bolz and I look at this is here: http://tratt.net/laurie/research/publications/papers/bolz_tr...
You can also download the full experimental suite and check out the particular benchmark(s) yourself. In my opinion, fijal has provided an accurate summary of the situation as we currently understand it.
It would be awesome to save the state of the JIT. It would also be awesome to export the type inference information.
How about an SHA2 hash of the source files?
A competent group can leave a dormant process that communicates over pipe for example and then the warmup issue disappears (it does not really appear for long running web application servers to start with)
Oh, well ...
The Lua port of Richards I've seen is naive and nobody within his right mind would write Lua code like that. As has been pointed out elsewhere, Richards is a very atypical benchmark and different implementation styles have very different performance characteristics.
In fact, PyPy wins ONLY on Richards by 1.2x and ONLY for a specific iteration count. But LuaJIT wins substantially on ALL OTHER BENCHMARKS where something is actually compiled (i.e. not regex stuff). Your paper clearly shows that in the appendix, but no matter what, you're still pointing out that one little result.
And, even worse, you're trying to draw conclusions from this outlier. I have absolutely no idea how you could possibly come to the conclusion that this makes PyPy more suitable for 'large scale' programs? And that LuaJIT is not? That's not just unscientific -- it's dishonest and disparaging.
Also, constantly repeating that 'hand-crafted JIT compilers' are somehow inferior to PyPy's overly complex approach is getting really old by now. LuaJIT runs on more architectures and more platforms than PyPy ever will. And it's vastly ahead in terms of optimizations. Now, please compare the timeframe all of this was implemented and the size of our teams. Ok, so what was the better deal, again?
And, no, the semantics of Lua and Python are not that far apart as you'd like everyone to believe. Dispatch is more complex in Python, but that can be taken care of in a central place in the code. Most library code can be written in terms of lower-level code. And so on. There are simple solutions for all of that. And none of that warrants a clumsy meta-tracing approach.
</rant>
I do, however, think you might be a little harsh on meta-tracing. It's an interesting approach, in no small part because it allows VMs to be written for languages other than Python. LuaJIT can be retargeted too (I've been impressed by Thomas Schilling's work), but I think it would be fair to say that it's harder work than writing a new RPython interpreter. In other words, IMHO, there is plenty of scope for both LuaJIT and RPython/PyPy to co-exist happily alongside each other.
Seriously, I would not try to say LuaJIT is worse than PyPy, ever. I picked richards, because it's the best benchmark from the set I know, while still being not a very good one. I'm all for saying "this is a bad benchmark".
I think you're putting words into my mouth. I did not say PyPy is better for large scale programs, I said we have more focus and I derive it from what we use as PyPy benchmarks. Since it's impossible to compare, because we lack programs we don't actually know, but my gut feeling would be that LuaJIT would be slightly faster on something large.
The problem with Python (and why we choose our approach instead of hand crafting the JIT) is that Python is a very large language, with lots of builtin modules and types interfering in very unobvious ways, while Lua is not. This is why we decided it makes more sense to go that way, we probably wouldn't do it for Lua. People tried to write a JIT by hand for Python and failed so far, I'm however fine with the explanation we're just too dumb to do that.
Overall I find your answer way too agressive - I didn't claim PyPy is better, I mostly pointed out to things I know and added it's very unscientific and that scientific approach is very hard to get. Also your attack on our meta-tracing is unfounded - Python is just a very very large language with strange corner cases and we have to somehow deal with it. Feel free to implement it differently, but I'm happy with our approach.
Would pypy be able to take advantage of Haswell's hardware transactional memory?
The HTM buffer is big enough if the goal is "only" to remove the GIL. It's going to be a 10-lines patch to CPython or PyPy (yes, really!). The GIL is often made into a big issue, but in my opinion removing it isn't the final solution to all multicore troubles in Python: that's just the first step.