I'm thankful they included a compiled language for comparison, because most of the time when I see Python benchmarks, they measure against other versions of Python. But "fast python" is an oxymoron and 3.14 doesn't seem to really change that, which I feel most people expected given the language hasn't fundamentally changed.
This isn't a bad thing; I don't think Python has to be or should be the fastest language in the world. But it's interesting to me seeing Python getting adopted for a purpose it wasn't suited for (high performance AI computing). Given how slow it is, people seem to think there's a lot of room for performance improvements. Take this line for instance:
> The free-threading interpreter disables the global interpreter lock (GIL), a change that promises to unlock great speed gains in multi-threaded applications.
No, not really. I mean, yeah you might get some speed gains, but the chart shows us if you want "great" speed gains you have two options: 1) JIT compile which gets you an order of magnitude faster or 2) switch to a static compiled language which gets you two orders of magnitude faster.
But there doesn't seem to be a world where they can tinker with the GIL or optimize python such that you'll approach JIT or compiled perf. If perf is a top priority, Python is not the language for you. And this is important because if they change Python to be a language that's faster to execute, they'll probably have to shift it away from what people like about it -- that it's a dynamic, interpreted language good for prototyping and gluing systems together.
It's pretty simple. Nobody wants to do ML R&D in C++.
Tensorflow is a C++ library with python bindings. Pytorch has supported a C++ interface for some time now, yet virtually nobody uses C++ for ML R&D.
The relationship between Python and C/C++ is the inverse of the usual backend/wrapper cases. C++ is the replaceable part of the equation. It's a means to an end. It's just there because python isn't fast enough. Nobody would really care if some other high perf language took its place.
Speed is important, but C++ is even less suited for ML R&D.
Tangential, but I practically owe my life to this guy. He wrote the flask mega tutorial in what I followed religiously to launch my first website. Then right before launch, in the most critical part of my entire application; piping a fragged file in flask. He answered my stackoverflow question, I put his fix live, and the site went viral. Here's the link for posterity's sake https://stackoverflow.com/a/34391304/4180276
Off-topic, but I absolutely loathe new Flask logo. Old one[0] has this vintage, crafty feel. And the new one[1] looks like it was made by a starving high schooler experimenting with WordArt.
For anyone else wondering whether to click to find what "fragged file" means: no, it's not about Quake and the linked page does not mention 'frag' at all. The question asks how to stream a file to the client in Flask as opposed to reading it all into memory at once and then sending it on. I figured as much (also because e.g. IP fragmentation) but first time I hear this alternative term for streaming
The accessibility of this material and also the broader python ecosystem is truly incredible. After reflecting on this recently, started finding ways to give back/donate/contribute.
Yet another appreciation story for Miguel’s mega tutorial. In 2017 I used it to create our wedding site and learn a bit of web dev (my background is in data science). To motivate me to actually do it I used the strategy the fund the then occurring refactoring of the tutorial. I am still very fond and proud of that first time I actually went and funded some open source effort, it gives you back more than you might expect
Very interesting post, thanks for putting it together.
Rust is indeed quite fast, I thought NodeJS was much better tbh., although it's not bad. I'd be interested to learn what's holding it back because I've seen many implementations where V8 can get C++-like performance (I mean it's C++ after all). Perhaps there's a lot of overhead in creating/destroying temporary objects.
tl;dr: Two orders of magnitude slower than Rust, so 2-3 orders slower than native. Python on a 2 GHz processor runs as fast as C on a 2-20 MHz processor.
> And this is a bit disappointing. At least for this test, the JIT interpreter did not produce any significant performance gains, so much that I had to double and triple check that I used a correctly built interpreter with this feature enabled. I do not know much about the internals of the new JIT compiler, but I'm wondering if it cannot deal with this heavily recursive function.
FWIW one thing that is worth calling out here is that the initial goal for JIT right now in Python is getting it relatively stable, functional, and more or less getting the initial implementation out there. It's not surprising at all that it's not faster.
I say this because I think the teams working on free-threaded and JIT python maybe could have done a better job publicly setting expectations.
Yeah honestly I don't really care about these benchmarks. Python isn't built for raw performance and that's totally fine! It's the number one choice for prototyping and can do so much, that's what actually matters. I think it's cool they're working on speed improvements though, means my prototype-to-production cycle gets a bit smoother lol.
honestly if the performance of the python interpreter has a big impact on your application's performance and that's something you care about - you're already doing things very wrong
That >2x performance increase over 3.9 in the first test is pretty impressive. A narrow use case for sure, but assuming you can leave your code completely alone and just have it run on a different interpreter via a few CLI commands, that's a nice bump.
Very nice post - it's good to see benchmarks done for humans.
For fun, I tried this in Raku:
(0, 1, *+* ... *)[40] #0.10s user 0.03s system 63% cpu 0.214 total
lol
Seriously, Python is doing great stuff to squeeze out performance from a scripting language. Realistically, Raku has fewer native libraries (although there is Inline::Python) and the compiler still has a lot of work to get the same degree of optimisation (although one day it could compare).
EDIT: for those who have commented, yes you are correct … this is a “cheat” and does not seek to state that Raku is faster than Python - as I said Raku still has a lot of work to do to catch up.
90 comments
[ 2.0 ms ] story [ 71.4 ms ] threadThis isn't a bad thing; I don't think Python has to be or should be the fastest language in the world. But it's interesting to me seeing Python getting adopted for a purpose it wasn't suited for (high performance AI computing). Given how slow it is, people seem to think there's a lot of room for performance improvements. Take this line for instance:
> The free-threading interpreter disables the global interpreter lock (GIL), a change that promises to unlock great speed gains in multi-threaded applications.
No, not really. I mean, yeah you might get some speed gains, but the chart shows us if you want "great" speed gains you have two options: 1) JIT compile which gets you an order of magnitude faster or 2) switch to a static compiled language which gets you two orders of magnitude faster.
But there doesn't seem to be a world where they can tinker with the GIL or optimize python such that you'll approach JIT or compiled perf. If perf is a top priority, Python is not the language for you. And this is important because if they change Python to be a language that's faster to execute, they'll probably have to shift it away from what people like about it -- that it's a dynamic, interpreted language good for prototyping and gluing systems together.
Tensorflow is a C++ library with python bindings. Pytorch has supported a C++ interface for some time now, yet virtually nobody uses C++ for ML R&D.
The relationship between Python and C/C++ is the inverse of the usual backend/wrapper cases. C++ is the replaceable part of the equation. It's a means to an end. It's just there because python isn't fast enough. Nobody would really care if some other high perf language took its place.
Speed is important, but C++ is even less suited for ML R&D.
Run as "python3 server.py -s 10000000 -n"
Seemingly.
Off-topic, but I absolutely loathe new Flask logo. Old one[0] has this vintage, crafty feel. And the new one[1] looks like it was made by a starving high schooler experimenting with WordArt.
[0] - https://upload.wikimedia.org/wikipedia/commons/3/3c/Flask_lo...
[1] - https://flask.palletsprojects.com/en/stable/_images/flask-na...
An once whimsical corner of web development has lost its charm due to arbitrary trends.
The accessibility of this material and also the broader python ecosystem is truly incredible. After reflecting on this recently, started finding ways to give back/donate/contribute.
one day of vibe coding
Edit: corrected typo in "typo".
That said, I wonder if GIL-less Python will one day enable GIL-less C FFI? That would be a big win that Python needs.
https://www.reddit.com/r/RedditDayOf/comments/7we430/donald_...
Rust is indeed quite fast, I thought NodeJS was much better tbh., although it's not bad. I'd be interested to learn what's holding it back because I've seen many implementations where V8 can get C++-like performance (I mean it's C++ after all). Perhaps there's a lot of overhead in creating/destroying temporary objects.
I say this because I think the teams working on free-threaded and JIT python maybe could have done a better job publicly setting expectations.
For fun, I tried this in Raku:
lolSeriously, Python is doing great stuff to squeeze out performance from a scripting language. Realistically, Raku has fewer native libraries (although there is Inline::Python) and the compiler still has a lot of work to get the same degree of optimisation (although one day it could compare).
EDIT: for those who have commented, yes you are correct … this is a “cheat” and does not seek to state that Raku is faster than Python - as I said Raku still has a lot of work to do to catch up.