Cool article, I think a lot of those issues are not Python specific so it's a good overview of whatever others can learn from a now 30 years old language! I think we'll probably go down the JS/TS route where another compiler (Pypy or mypyc or something else) will work alongside CPython but I don't see Python4 happening.
It isn't. There are many things Python isn't up to the task. However, it has been around forever, and some influential niche verticals like cyber security Python was as or more useful than native tooling, and works on multiple platforms.
Most of the time you are waiting on a human or at least something other than the cpu. Most of the time more time is spent by the programmer writing the code than all the users combined waiting for the program to run.
between those two, most often performance is just fine to trade off.
Performance is worthless if the code isn't correct. It's easier to write correct code reasonably quickly in Python in simple cases (integers don't overflow like in C, don't wrap around like in C#, no absurd implicit conversions like in other scripting languages).
Also you don't need code to be fast a lot of the time. If you just need some number crunching that is occasionally run by a human, taking a whole second is fine. Pretty good replacement for shell scripting too.
I don't think anyone aware of this thinks it's a good tradeoff.
The more interesting question is why the tradeoff was made in the first place.
The answer is, it's relatively easy for us to see and understand the impact of these design decisions because we've been able to see their outcomes over the last 20+ years of Python. Hindsight is 20/20.
Remember that Python was released in 1991, before even Java. What we knew about programming back then vs what we know now is very different.
Oh and also, these tradeoffs are very hard to make in general. A design decision that you may think is irrelevant at the time may in fact end up being crucial to performance later on, but by that point the design is set in stone due to backwards compatibility.
I think an important bit of context here is that computers are very, very good at speculative happy-path execution.
The examples in the article seem gloomy: how could a JIT possibly do all the checks to make sure the arguments aren’t funky before adding them together, in a way that’s meaningfully better than just running the interpreter? But in practice, a JIT can create code that does these checks, and modern processors will branch-predict the happy path and effectively run it in parallel with the checks.
JavaScript, too, has complex prototype chains and common use of boxed objects - but v8 has made common use cases extremely fast. I’m excited for the future of Python.
The main problem is when the optimizations silently fail because of seemingly innocent changes and suddenly your performance tanked 10x. This is a problem with any language really (CPU cache misses are a thing afterall and many non-dynamic languages have boxed objects) but it is a much, much worse in dynamic languages like Python, JS and Ruby.
Most of the time it doesn't matter, most high-throughput python code just invokes C/C++ where these concerns are not as big of a problem. Most JS code just invokes C/C++ browser DOM objects. As long as the hot-path is not in those languages you are not at such high risk of "innocent change tanked performance"
Even server-side most JS/Python/Ruby code is just simple HTTP stack handlers and invoking databases and shuffling data around. And often large part of the process of handling a request (encoding JSON/XML/etc, parsing HTTP messages, etc) can be written in lower-level languages.
To be slightly flip, we could say that the Lisp Machine CISC-supports-language full stack design philosophy lives on in how massive M-series reorder buffers and ILP supports JavaScriptCore.
That makes it so that in absolute terms, Python is not as slow as you might naively expect.
But we don't measure programming language performance in absolute terms. We measure them in relative terms, generally against C. And while your Python code is speculating about how this Python object will be unboxed, where its methods are, how to unbox its parameters, what methods will be called on those, etc., compiled code is speculating on actual code the programmer has written, running that in parallel, such that by the time the Python interpreter is done speculating successfully on how some method call will resolve with actual objects the compiled code language is now done with ~50 lines of code of similar grammatical complexity. (Which is a sloppy term, since this is a bit of a sloppy conversation, but consider a series "p.x = y"-level statements in Python versus C as the case I'm looking at here.)
There's no way around it. You can spend your amazingly capable speculative parallel CPU on churning through Python interpretation or you can spend it on doing real work, but you can't do both.
After all, the interpreter is just C code too. It's not like it gets access to special speculation opcodes that no other program does.
Although JS supports prototype mutations, the with operator and other constructs that make optimization harder, typical JS code does not use that. Thus JIT can add few checks for presence of problematic constructions to direct it to a slow path while optimizing not particularly big set of common patterns. And then the JS JIT does not need to care much about calling arbitrary native code as the browser internals can be adjusted/refactored to tune to JIT needs.
With Python that does not work. There are simply more optimization-unfriendly constructs and popular libraries use those. And Python calls arbitrary C libraries with fixed ABI.
So optimizing Python is inherently more difficult.
> but v8 has made common use cases extremely fast. I’m excited for the future of Python.
Isn't v8 still entirely single threaded with limited message passing? Python just went through a lot of work to make multithreaded code faster, it would be disappointing if it had to scrap threading entirely and fall back to multiprocessing on shared memory in order to match v8.
I wonder if branch prediction can still hide the performance loss when the happy path checks become large/complex. Branch prediction is a very low level optimisation. And if the predictor is right you don't get everything for free. The CPU must still evaluate the condition, which takes resources, albeit it's no longer on the critical path. However I'd think the CPU would stall if it got too far ahead of the condition execution (ultimately all the code must execute before the program completes). Perhaps given the nature of Python, the checks would be so complex that in a tight loop they'd exert significant resource pressure?
There's an obvious answer - run everything on GPUs. Each speculative branch runs in parallel on its own core, and you add a layer of super-fast branch switching when a branch runs into a problem.
Given the current state of computing, I am unable to state definitively if this suggestion is satire.
A more careful reading of the article is required.
The first myth is "Python is not slow" - it is debunked, it is slow.
The second myth is ""it's just a glue language / you just need to rewrite the hot parts in C/C++" - it is debunked, just rewriting stuff in C/Rust does not help.
The third myth is " Python is slow because it is interpreted" - it is debunked, it is not slow only because it is interpreted.
And yet, most of what people end up doing ends up being effectively OS and application scripting. Most ML projects are really just setting up a pipeline and telling the computer to go and run it. Cloud deployments are "take this yaml and transform it some other yaml". In as much as I don't want to use Fortran to parse a yaml file, I don't really want to write an OS (or a database) in Python. Even something like django is mostly deferring off tasks to faster systems, and is really about being a DSL-as-programming-language while still being able to call out to other things (e.g. ML code).
Feel like Mojo is worth a shoutout in this context https://www.modular.com/mojo Solves the issue of having a superset of Python in syntax where "fn" instead of "def" functions are assumed static typed and compilable with Numba style optimisations.
"Rewrite the hot path in C/C++" is also a landmine because how inefficient the boundary crossing is. so you really need "dispatch as much as possible at once" instead of continuously calling the native code
Python as a language will likely never have a "fast" implementation and still be Python. It is way too dynamic to be predictable from the code alone or even an execution stream in a way that allows you to simplify the actual code that will be executed at runtime either through AOC or JIT. The language is itself is also quite large in terms of syntax and built-in capability at this point which makes new feature-conplete implementations that don't make major trade offs quite challenging. Given how capable LLMs are at translating code, it seems like the perfect time to build a language with similar syntax, but better scoped behavior, stricter rules around typing, and tooling to make porting code and libraries automated and relatively painless. What would existing candidates be and why won't they work as a replacement?
The primary focus here is good and something I hadn't considered: python memory being so dynamic leads to poor cache locality. Makes sense. I will leave that to others to dig into.
That aside, I was expecting some level of a pedantic argument, and wasn't disappointed by this one:
"A compiler for C/C++/Rust could turn that kind of expression into three operations: load the value of x, multiply it by two, and then store the result. In Python, however, there is a long list of operations that have to be performed, starting with finding the type of p, calling its __getattribute__() method, through unboxing p.x and 2, to finally boxing the result, which requires memory allocation. None of that is dependent on whether Python is interpreted or not, those steps are required based on the language semantics."
The problem with this argument is the user isn't trying to do these things, they are trying to do multiplication, so the fact that the lang. has to do all things things in the end DOES mean it is slow. Why? Because if these things weren't done, the end result could still be achieved. They are pure overhead, for no value in this situation. Iow, if Python had a sufficiently intelligent compiler/JIT, these things could be optimized away (in this use case, but certainly not all). The argument is akin to: "Python isn't slow, it is just doing a lot of work". That might be true, but you can't leave it there. You have to ask if this work has value, and in this case, it does not.
By the same argument, someone could say that any interpreted language that is highly optimized is "fast" because the interpreter itself is optimized. But again, this is the wrong way to think about this. You always have to start by asking "What is the user trying to do? And (in comparison to what is considered a fast language) is it fast to compute?". If the answer is "no", then the language isn't fast, even if it meets the expected objectives. Playing games with things like this is why users get confused on "fast" vs "slow" languages. Slow isn't inherently "bad", but call a spade a spade. In this case, I would say the proper way to talk about this is to say: "It has a fast interpreter". The last word tells any developer with sufficient experience what they need to know (since they understand statically compiled/JIT and interpreted languages are in different speed classes and shouldn't be directly compared for execution speed).
> They are pure overhead, for no value in this situation. Iow, if Python had a sufficiently intelligent compiler/JIT, these things could be optimized away (in this use case, but certainly not all).
I don't know Python so well as to propose any meaningful contribution, but it seems to me that most issues would be mitigated by a sort of "final" statement or qualifier, that prohibits any further changes to the underlying data structure, thus enabling all the nice optimizations, tricks and shortcuts that compilers and interpreters can't afford when data is allowed to change shape under their feet.
I know I am going to get some hate for this from the "Python-stans" but..."python" and "performance" should never be associated with each other, and same for any scripting/interpreted programming language. Especially if it has a global interpreter lock.
While performance (however you may mean that) is always a worthy goal, you may need to question your choice of language if you start hitting performance ceilings.
As the saying goes - "Use the right tool for the job." Use case should dictate tech choices, with few exceptions.
Ok, now that I have said my piece, now you can down vote me :)
Again and again, the most important question is "why?" not "how?". Python isn't made to be fast. If you wanted a language that can go fast, you needed to build it into the language from the start: give developers tools to manage memory layout, give developers tools to manage execution flow, hint the compiler about situations that present potential for optimization, restrict dispatch and polymorphism, restrict semantics to fewer interpretations.
Python has none of that. It's a hyper-bloated language with extremely poor design choices all around. Many ways of doing the same thing, many ways of doing stupid things, no way of communicating programmer's intention to the compiler... So why even bother? Why not use a language that's designed by a sensible designer for this specific purpose?
The news about performance improvements in Python just sound to me like spending useful resources on useless goals. We aren't going forward by making Python slightly faster and slightly more bloated, we just make this bad language even harder to get rid of.
In the "dynamic" section, it's much worse than the author outlines. You can't even assume that the constant named "10" will point to a value which behaves like you expect the number 10 to behave.
Python and other high-level languages may actually decrease in popularity with better LLMs. If you are not the one programming it, might as well do it in a more performant language from the start.
LLMs will make most languages irrelevant indeed, just like most developers have no idea about the Assembly/machine code that the JIT/AOT compilers of their favourite programming language happen to generate.
Eventually LLMs might even generate executables directly.
Slow or fast ultimately matter in the context for which you need to use it. Perhaps these are only myths and fairly tales for an incredibly small subset of people who value execution speed as the highest priority, but choose to use Python for implementation.
56 comments
[ 3.5 ms ] story [ 81.7 ms ] threadWhy do people think it's a good trade-off?
between those two, most often performance is just fine to trade off.
Also you don't need code to be fast a lot of the time. If you just need some number crunching that is occasionally run by a human, taking a whole second is fine. Pretty good replacement for shell scripting too.
The more interesting question is why the tradeoff was made in the first place.
The answer is, it's relatively easy for us to see and understand the impact of these design decisions because we've been able to see their outcomes over the last 20+ years of Python. Hindsight is 20/20.
Remember that Python was released in 1991, before even Java. What we knew about programming back then vs what we know now is very different.
Oh and also, these tradeoffs are very hard to make in general. A design decision that you may think is irrelevant at the time may in fact end up being crucial to performance later on, but by that point the design is set in stone due to backwards compatibility.
Even if you have to stick to CPython, Numba, Pythran etc, can give you amazing performance for minimal code changes.
The examples in the article seem gloomy: how could a JIT possibly do all the checks to make sure the arguments aren’t funky before adding them together, in a way that’s meaningfully better than just running the interpreter? But in practice, a JIT can create code that does these checks, and modern processors will branch-predict the happy path and effectively run it in parallel with the checks.
JavaScript, too, has complex prototype chains and common use of boxed objects - but v8 has made common use cases extremely fast. I’m excited for the future of Python.
Most of the time it doesn't matter, most high-throughput python code just invokes C/C++ where these concerns are not as big of a problem. Most JS code just invokes C/C++ browser DOM objects. As long as the hot-path is not in those languages you are not at such high risk of "innocent change tanked performance"
Even server-side most JS/Python/Ruby code is just simple HTTP stack handlers and invoking databases and shuffling data around. And often large part of the process of handling a request (encoding JSON/XML/etc, parsing HTTP messages, etc) can be written in lower-level languages.
But we don't measure programming language performance in absolute terms. We measure them in relative terms, generally against C. And while your Python code is speculating about how this Python object will be unboxed, where its methods are, how to unbox its parameters, what methods will be called on those, etc., compiled code is speculating on actual code the programmer has written, running that in parallel, such that by the time the Python interpreter is done speculating successfully on how some method call will resolve with actual objects the compiled code language is now done with ~50 lines of code of similar grammatical complexity. (Which is a sloppy term, since this is a bit of a sloppy conversation, but consider a series "p.x = y"-level statements in Python versus C as the case I'm looking at here.)
There's no way around it. You can spend your amazingly capable speculative parallel CPU on churning through Python interpretation or you can spend it on doing real work, but you can't do both.
After all, the interpreter is just C code too. It's not like it gets access to special speculation opcodes that no other program does.
With Python that does not work. There are simply more optimization-unfriendly constructs and popular libraries use those. And Python calls arbitrary C libraries with fixed ABI.
So optimizing Python is inherently more difficult.
Isn't v8 still entirely single threaded with limited message passing? Python just went through a lot of work to make multithreaded code faster, it would be disappointing if it had to scrap threading entirely and fall back to multiprocessing on shared memory in order to match v8.
Given the current state of computing, I am unable to state definitively if this suggestion is satire.
The first myth is "Python is not slow" - it is debunked, it is slow.
The second myth is ""it's just a glue language / you just need to rewrite the hot parts in C/C++" - it is debunked, just rewriting stuff in C/Rust does not help.
The third myth is " Python is slow because it is interpreted" - it is debunked, it is not slow only because it is interpreted.
But honestly the thing that makes any of my programs slow is network calls. And there a nice async setup goes a long way. And then k8 for the scaling.
That aside, I was expecting some level of a pedantic argument, and wasn't disappointed by this one:
"A compiler for C/C++/Rust could turn that kind of expression into three operations: load the value of x, multiply it by two, and then store the result. In Python, however, there is a long list of operations that have to be performed, starting with finding the type of p, calling its __getattribute__() method, through unboxing p.x and 2, to finally boxing the result, which requires memory allocation. None of that is dependent on whether Python is interpreted or not, those steps are required based on the language semantics."
The problem with this argument is the user isn't trying to do these things, they are trying to do multiplication, so the fact that the lang. has to do all things things in the end DOES mean it is slow. Why? Because if these things weren't done, the end result could still be achieved. They are pure overhead, for no value in this situation. Iow, if Python had a sufficiently intelligent compiler/JIT, these things could be optimized away (in this use case, but certainly not all). The argument is akin to: "Python isn't slow, it is just doing a lot of work". That might be true, but you can't leave it there. You have to ask if this work has value, and in this case, it does not.
By the same argument, someone could say that any interpreted language that is highly optimized is "fast" because the interpreter itself is optimized. But again, this is the wrong way to think about this. You always have to start by asking "What is the user trying to do? And (in comparison to what is considered a fast language) is it fast to compute?". If the answer is "no", then the language isn't fast, even if it meets the expected objectives. Playing games with things like this is why users get confused on "fast" vs "slow" languages. Slow isn't inherently "bad", but call a spade a spade. In this case, I would say the proper way to talk about this is to say: "It has a fast interpreter". The last word tells any developer with sufficient experience what they need to know (since they understand statically compiled/JIT and interpreted languages are in different speed classes and shouldn't be directly compared for execution speed).
Hence, Numba.
While performance (however you may mean that) is always a worthy goal, you may need to question your choice of language if you start hitting performance ceilings.
As the saying goes - "Use the right tool for the job." Use case should dictate tech choices, with few exceptions.
Ok, now that I have said my piece, now you can down vote me :)
Python has none of that. It's a hyper-bloated language with extremely poor design choices all around. Many ways of doing the same thing, many ways of doing stupid things, no way of communicating programmer's intention to the compiler... So why even bother? Why not use a language that's designed by a sensible designer for this specific purpose?
The news about performance improvements in Python just sound to me like spending useful resources on useless goals. We aren't going forward by making Python slightly faster and slightly more bloated, we just make this bad language even harder to get rid of.
Eventually LLMs might even generate executables directly.