CPython 3.13 went further with an experimental copy-and-patch JIT compiler -- a lightweight JIT that stitches together pre-compiled machine code templates instead of generating code from scratch. It's not a full optimizing JIT like V8's TurboFan or a tracing JIT like PyPy's;
Good news. Python 3.15 adapts Pypy tracing approach to JIT and there are real performance gains now:
> The real story is that Python is designed to be maximally dynamic -- you can monkey-patch methods at runtime, replace builtins, change a class's inheritance chain while instances exist -- and that design makes it fundamentally hard to optimize. ...
> 4 bytes of number, 24 bytes of machinery to support dynamism. a + b means: dereference two heap pointers, look up type slots, dispatch to int.__add__, allocate a new PyObject for the result (unless it hits the small-integer cache), update reference counts.
Would Python be a lot less useful without being maximally dynamic everywhere? Are there domains/frameworks/packages that benefit from this where this is a good trade-off?
I can't think of cases in strong statically typed languages where I've wanted something like monkey patching, and when I see monkey patching elsewhere there's often some reasonable alternative or it only needs to be used very rarely.
The dynamism exists to support the object model. That's the actual dependency. Monkey-patching, runtime class mutation, vtable dispatch. These aren't language features people asked for. They're consequences of building everything on mutable objects with identity.
Strip the object model. Keep Python.
You get most of the speed back without touching a compiler, and your code gets easier to read as a side effect.
I built a demo: Dishonest code mutates state behind your back; Honest code takes data in and returns data out. Classes vs pure functions in 11 languages, same calculation. Honest Python beats compiled C++ and Swift on the same problem. Not because Python is fast, but because the object model's pointer-chasing costs more than the Python VM overhead.
Don't take my word for it. It's dockerized and on GitHub. Run it yourself: honestcode.software, hit the Surprise! button.
I've been in the pandas (and now polars world) for the past 15 years. Staying in the sandbox gets most folks good enough performance. (That's why Python is the language of data science and ML).
I generally teach my clients to reach for numba first. Potentially lots of bang for little buck.
One overlooked area in the article is running on GPUs. Some numpy and pandas (and polars) code can get a big speedup by using GPUs (same code with import change).
Python is perfect as a "glue" language. "Inner Loops" that have to run efficiently is not where it shines, and I would write them in C or C++ and patch them with Python for access to the huge library base.
This is the "two language problem" ( I would like to hear from people who extensively used Julia by the way, which claims to solve this problem, does it really ?)
I have used Julia for my main language for years. Yes, it really does solve the two language problem. It really is as fast as C and as expressive as Python.
It then gives you a bunch of new problems. First and foremost that you now work in a niche language with fewer packages and fewer people who can maintain the code. Then you get the huge JIT latency. And deployment issues. And lack of static tooling which Rust and Python have.
For me, as a research software engineer writing performance sensitive code, those tradeoffs are worth it. For most people, it probably isn’t. But if you’re the kind of person who cares about the Python optimization ladder, you should look into Julia. It’s how I got hooked.
As a sibling comment mentions, yes it does. Just don’t expect to have code that runs as fast as C without some effort put into it. You still need to write your program in a static enough way to obtain those speed. It’s not the easiest thing in the world, since the tooling is, yes, improving but is still not there yet.
If you then want to access fully trimmed small executables then you have to start writing Julia similarly to how you write rust.
To me the fact that this is even possible blows my mind and I have tons of fun coding in it. Except when precompiling things. That is something that really needs to be addressed.
A personal opinion: I would much prefer to read the rough, human version of this article than this AI-polished version. I'm interested in the content and the author clearly put thought and effort into it, but I'm constantly thrown out of it by the LLM smell. (I'm also a bit mad that `--` is now on the em dash treadmill and will soon be unusable.)
I'm not just saying this to vent. I honestly wonder if we could eventually move to a norm where people publish two versions of their writing and allow the reader to choose between them. Even when the original is just a set of notes, I would personally choose to make my own way through them.
Surprised Python is only 21x slower than C for tree traversal stuff. In my experience that's one of the most painful places to use Python. But maybe that's because I use numpy automatically when simple arrays are involved, and there's no easy path for trees.
Kudos for going through all the existing JIT approaches, instead of reaching for rewrite into X straight away.
However if Rust with PyO3 is part of the alternatives, then Boost.Python, cppyy, and pybind11 should also be accounted for, given their use in HPC and HFT integrations.
Significant AI smell in this write up. As a result, my current reflex is to immediately stop reading. Not judgement on the actual analysis and human effort which went in. It’s just that the other context is missing.
I have the same issue now. It's especially annoying when it happens while reading a "serious" publication like a newspaper or long form magazine. Whether it was because an AI wrote it or "real" writers have spent so much time reading AI slop they've picked up the same style is kinda by the by. It all reads to me like SEO, which was the slop template that LLMs took their inspiration from, apparently. It just flattens language into the most exhausting version of it, where you need to try to subconsciously blank out all the unnecessary flourishes and weird hype phrases to try figure out what actually is trying to be said. I guess humans who learn to ignore it might to do better in this brave new world, but it's definitely annoying that humans are being forced to adapt to machines instead of the other way around.
What is the point of your post? I find it increasingly tedious to read comments about alledged AI use under almost every post. It's like complaining that you didn't want to read the submission because you didn't like their font or website design.
I think almost everyone here agrees they don't want to read AI slop, but this submission clearly wasn't that as you admit yourself.
Shockingly good article — correct identification of the root cause of performance issues being excessive dynamism and ranking of the solutions based on the value/effort ratio. Excellent taste. Will keep this in my back pocket as a quick Python optimization reference.
It's just somewhat unfortunate that I have to question every number and fact presented since the writing was clearly at least somewhat AI-assisted with the author seemingly not being upfront about that at all.
> the reference implementation of language is slow
Despite its content, this blogpost also pushes this exact "language slow" thinking in its preamble. I don't think nearly enough people read past introductions for that to be a responsible choice or a good idea.
The only thing worse than this is when Python specifically is outright taught (!) as an "interpeted language", as if an implementation-detail like that was somehow a language property. So grating.
>The usual suspects are the GIL, interpretation, and dynamic typing. All three matter, but none of them is the real story. The real story is that Python is designed to be maximally dynamic -- you can monkey-patch methods at runtime, replace builtins, change a class's inheritance chain while instances exist -- and that design makes it fundamentally hard to optimize.
ok I guess the harder question is. Why isn't python as fast as javascript?
Beyond the economic arguments, there’s a lot in JS that actually makes it a lot easier: almost all of the operators can only return a subset of the types and cannot be overridden (e.g., the binary + operator in JS can only return a string or a number primitive), the existence of like string and number primitives dramatically reduce the amount of dynamic behaviour they can have, only proxy objects can exhibit the same amount of dynamism as arbitrary Python ones (and thus only they pay the performance cost)…
The more real answer is that python's primary usage is a glue language. It has to be able to interface with various C libraries, and to make the interfaces even more ergonomic, they exposed several internal details on how code is evaluated that libraries make use of (e.g. you can increment/decrement a ref counted python object's counter from C).
This pretty much makes it impossible to change many of the internal details, and to significantly optimize it.
If we remove this requirement, we get the alternative runtimes and if you check e.g. GraalPy, it has the same order of performance as JS, so your intuition is right. It's just that you have to drop supporting a good chunk of what people use Python for which is obviously a no go for most applications. (Note: GraalPy can actually also run some C libraries and in this case can cross-optimize across python and C!)
I must admit that I'm amused by the people who find the writeup useful but are turned off by the AI "smell". And look forward to the day when all valued content reeks of said "smell"; let's see what detractors-for-no-good-reason do then (yes I'm a bit ticked by the attitude).
It's partly just a matter of taste; we can disagree on whether that's a good reason, but I'd be surprised if there were no writing styles that you personally find offputting.
The LLM smell is also a signal of low effort, and a signal that we as readers can't rely on our usual heuristics for judging credibility. The whole thing with LLMs is that they're great at producing polished, plausible-looking outputs, but they're still prone to bullshitting and making errors that don't match the usual human patterns. (And of course they're a great tool for churning out human-initiated disinformation.) If you don't have any kind of immune response against the LLM smell, I reckon you're probably absorbing more bs than you realise.
Is low effort really a valuable signal though? Or is it what's actually in the content that's valuable? Like here readers are literally saying that they found the content valuable "but AI smell". Why is there a "but"? Would there be a similar issue if the author had contracted a human assistant to do X? Definitely not, and I see no reason why the treatment should be different for AI.
> Missing @cython.cdivision(True) inserts a zero-division check before every floating-point divide in the inner loop. Millions of branches that are never taken.
I thought never taken branches were essentially free. Does this mean something in the loop is messing with the branch predictor?
From when I was working on optimizing one or two things with Cython years ago, it wasn’t per-se the branch cost that hurt: it was impeding the compiler from various loop optimisations, potentially being the impediment from going all the way to auto-vectorisation.
json.loads is something you don't want to use in a loop if you care for performance at all. Just simple using orjson can give you 3x speed without the need to change anything.
The replacement of emdashes with double hyphens here is almost insulting. A look through the blog history suggests that the author has no issue writing in English normally, and nothing seems really off about the actual findings here (or even the speculation about causes etc.), so I really can't understand the motivation for LLM-generated prose. (The author's usual writing style appears to have some arguable LLM-isms, but they make a lot more sense in context and of course those patterns had to come from somewhere. The overall effect is quite different.)
Edit: it's strange to get downvoted while also getting replies that agree with me and don't seem to object.
(Also, I thought it wasn't supposed to be possible to edit after getting a reply?)
In practice the ladder has two rungs for me. Write it in Python with numpy/scipy doing the heavy lifting, and if that's not enough, rewrite the hot path in C. The middle steps always felt like they added complexity without fully solving the problem.
The JIT work kenjin4096 describes is really promising though. If the tracing JIT in 3.15 actually sticks, a lot of this ladder just goes away for common workloads.
Great write up and recognisable performance. For a pipeline with many (~50) build dependencies unfortunately switching interpreter or experimenting with free threading is not an easy route as long as packages are not available (which is completely understandable).
I’m not one of these rewrite in Rust types, but some isolated jobs are just so well sorted for full control system programming that the rust delegation is worth the investment imo.
Another part worth investigating for IO bound pipelines is different multiprocessing techniques. We recently got a boost from using ThreadPoolExecutor over standard multiprocessing, and careful profiling to identify which tasks are left hanging and best allocated its own worker. The price you pay though is shared memory, so no thread safety, which only works if your pipeline can be staggered
49 comments
[ 3.1 ms ] story [ 71.7 ms ] threadhttps://github.com/python/cpython/issues/139109
https://doesjitgobrrr.com/?goals=5,10
> 4 bytes of number, 24 bytes of machinery to support dynamism. a + b means: dereference two heap pointers, look up type slots, dispatch to int.__add__, allocate a new PyObject for the result (unless it hits the small-integer cache), update reference counts.
Would Python be a lot less useful without being maximally dynamic everywhere? Are there domains/frameworks/packages that benefit from this where this is a good trade-off?
I can't think of cases in strong statically typed languages where I've wanted something like monkey patching, and when I see monkey patching elsewhere there's often some reasonable alternative or it only needs to be used very rarely.
Strip the object model. Keep Python.
You get most of the speed back without touching a compiler, and your code gets easier to read as a side effect.
I built a demo: Dishonest code mutates state behind your back; Honest code takes data in and returns data out. Classes vs pure functions in 11 languages, same calculation. Honest Python beats compiled C++ and Swift on the same problem. Not because Python is fast, but because the object model's pointer-chasing costs more than the Python VM overhead.
Don't take my word for it. It's dockerized and on GitHub. Run it yourself: honestcode.software, hit the Surprise! button.
I've been in the pandas (and now polars world) for the past 15 years. Staying in the sandbox gets most folks good enough performance. (That's why Python is the language of data science and ML).
I generally teach my clients to reach for numba first. Potentially lots of bang for little buck.
One overlooked area in the article is running on GPUs. Some numpy and pandas (and polars) code can get a big speedup by using GPUs (same code with import change).
This is the "two language problem" ( I would like to hear from people who extensively used Julia by the way, which claims to solve this problem, does it really ?)
It then gives you a bunch of new problems. First and foremost that you now work in a niche language with fewer packages and fewer people who can maintain the code. Then you get the huge JIT latency. And deployment issues. And lack of static tooling which Rust and Python have.
For me, as a research software engineer writing performance sensitive code, those tradeoffs are worth it. For most people, it probably isn’t. But if you’re the kind of person who cares about the Python optimization ladder, you should look into Julia. It’s how I got hooked.
If you then want to access fully trimmed small executables then you have to start writing Julia similarly to how you write rust.
To me the fact that this is even possible blows my mind and I have tons of fun coding in it. Except when precompiling things. That is something that really needs to be addressed.
I'm not just saying this to vent. I honestly wonder if we could eventually move to a norm where people publish two versions of their writing and allow the reader to choose between them. Even when the original is just a set of notes, I would personally choose to make my own way through them.
[0]: https://www.muna.ai/ [1]: https://docs.muna.ai/predictors/create
However if Rust with PyO3 is part of the alternatives, then Boost.Python, cppyy, and pybind11 should also be accounted for, given their use in HPC and HFT integrations.
I think almost everyone here agrees they don't want to read AI slop, but this submission clearly wasn't that as you admit yourself.
It's just somewhat unfortunate that I have to question every number and fact presented since the writing was clearly at least somewhat AI-assisted with the author seemingly not being upfront about that at all.
> looks inside
> the reference implementation of language is slow
Despite its content, this blogpost also pushes this exact "language slow" thinking in its preamble. I don't think nearly enough people read past introductions for that to be a responsible choice or a good idea.
The only thing worse than this is when Python specifically is outright taught (!) as an "interpeted language", as if an implementation-detail like that was somehow a language property. So grating.
ok I guess the harder question is. Why isn't python as fast as javascript?
This pretty much makes it impossible to change many of the internal details, and to significantly optimize it.
If we remove this requirement, we get the alternative runtimes and if you check e.g. GraalPy, it has the same order of performance as JS, so your intuition is right. It's just that you have to drop supporting a good chunk of what people use Python for which is obviously a no go for most applications. (Note: GraalPy can actually also run some C libraries and in this case can cross-optimize across python and C!)
It's partly just a matter of taste; we can disagree on whether that's a good reason, but I'd be surprised if there were no writing styles that you personally find offputting.
The LLM smell is also a signal of low effort, and a signal that we as readers can't rely on our usual heuristics for judging credibility. The whole thing with LLMs is that they're great at producing polished, plausible-looking outputs, but they're still prone to bullshitting and making errors that don't match the usual human patterns. (And of course they're a great tool for churning out human-initiated disinformation.) If you don't have any kind of immune response against the LLM smell, I reckon you're probably absorbing more bs than you realise.
Is low effort really a valuable signal though? Or is it what's actually in the content that's valuable? Like here readers are literally saying that they found the content valuable "but AI smell". Why is there a "but"? Would there be a similar issue if the author had contracted a human assistant to do X? Definitely not, and I see no reason why the treatment should be different for AI.
Some nuance: try transpiling to a garbage collected rust like language with fast compilation until you have millions of users.
Also use a combination of neural and deterministic methods to transpile depending on the complexity.
> Missing @cython.cdivision(True) inserts a zero-division check before every floating-point divide in the inner loop. Millions of branches that are never taken.
I thought never taken branches were essentially free. Does this mean something in the loop is messing with the branch predictor?
Edit: it's strange to get downvoted while also getting replies that agree with me and don't seem to object.
(Also, I thought it wasn't supposed to be possible to edit after getting a reply?)
The JIT work kenjin4096 describes is really promising though. If the tracing JIT in 3.15 actually sticks, a lot of this ladder just goes away for common workloads.
I’m not one of these rewrite in Rust types, but some isolated jobs are just so well sorted for full control system programming that the rust delegation is worth the investment imo.
Another part worth investigating for IO bound pipelines is different multiprocessing techniques. We recently got a boost from using ThreadPoolExecutor over standard multiprocessing, and careful profiling to identify which tasks are left hanging and best allocated its own worker. The price you pay though is shared memory, so no thread safety, which only works if your pipeline can be staggered