When you can easily drop down to low-levels and write code in C or even Assembly in Python, why bother using any other language? I'm pretty confident at this point that Python is just going to eclipse anything else, I've been programming for years now and aside from using JS in browser I find it baffling that people would use anything but Python or C.
Something has to happen regarding Python performance.
What could it be?
Can CPython become faster? Could the devs take a look at how PHP and Javascript did it to make their runtimes so fast?
Can the Python language get extended so that you can define the bottlencks of your code in another language? Maybe in C? PHP? Javascript? Webassembly? Python bytecode?
Can a new runtime take over and everybody settle on it? Which one could it be?
[disclaimer: I have contributed to a couple different Python runtimes and employed to work on Cinder]
> Can CPython become faster?
Yes. There are several people (including a team by Guido and Mark Shannon) working on making CPython faster in-tree. There are others, including the folks on Cinder and Pyjion, who are working on extensions to CPython out of tree.
> Could the devs take a look [...]
It's no secret how they made their runtimes fast. It's all -- more or less -- implementations of the original Smalltalk-80 paper and all that fell out of it.
> Can the Python language get extended so that you can define the bottlencks of your code in another language?
C extensions are already available in Python, and have been for quite some time. In fact, I would argue they are a little bit part of the problem: there are so many C extensions that runtime optimization efforts don't have as much effect as people would like; the slow stuff is already in C.
> Can a new runtime take over and everybody settle on it?
There are a lot of "competing" Python runtimes that serve different use-cases. Some use PyPy. Some use Jython. Some use Cinder. etc. Though everyone seems to ask "why not just use PyPy?", people forget to consider that everyone has different constraints that PyPy does not necessarily satisfy.
McCarthy is generally credited with the first JIT for his early LISP work, but SmallTalk and SmallTalk-inspired languages were involved in a lot of the pioneering work related to modern JITs. My understanding is that McCarthy's JIT generated machine code as source code was first loaded, not when it was first invoked. As far as I know, the earliest VMs to wait until a function's first invocation to JIT the function were SmallTalk VMs.
The Self language by David Ungar and others was an expansion of some of the SmallTalk ideas. If I remember correctly, Self is where polymorphic inline caching originated. Sun acqui-hired Urs Holzle (who did his PhD under Ungar), David Griswald, Gilad Bracha, and Lars Bak for their StrongTalk (statically typed dialect of SmallTalk) hybrid bytecode interpreter/JIT, which was the basis for the HotSpot JVM.
Lars Bak later created the original V8 JavaScript JIT for Chrome. When I was on the Google web search indexing team, I did a bit of work running a pre-release V8 in the indexing pipeline (to analyze text and formatting changes made to pages by JavaScript). If I remember correctly, at that time V8 wasn't a hybrid interpreter/JIT, but did quick-and-dirty generation of x86 machine code when a function was first invoked, and leaned heavily on polymorphic inline caching. My understanding is that early V8 was rather similar to the Self VM.
I believe ideas like tracing JITs originated a bit further from the SmallTalk ecosystem, but a lot of modern JIT ideas and techniques came from early SmallTalk and related research.
I don't remember any Lisp work being considered an example of a JIT, the Deutsch/Schiffman paper was felt to be a big breakthrough when it was published.
John Aycock credits McCarthy's Lisp work in "A brief history of just-in-time"[0] Though, as I mentioned, I believe the native code was generated when the module was loaded. These days, we generally consider JIT compilation to be compilation that happens at first invocation (or after, in the case of hybrid interpretation/JIT), so it doesn't precisely fit the modern notion of JIT compilation.
You will recognize some of the V8 names on those papers.
Additionally, SELF JIT was repurposed as Hotspot by Java 1.2.
Although Java looks like C++ from syntax point of view, it actually is closer to Smalltalk and Objective-C in runtime semantics, thus it could profit a lot from that JIT research as well, even though it was originally targeted to dynamic languages.
> C extensions are already available in Python, and have been for quite some time.
And C extensions are an API, you don’t have to write them in C e.g. cython generates “C extensions”.
Furthermore there are other micro-hooks like numba ehich integrate rather easy and can provide great bang for your buck in restricted numerical contexts.
I had to look up what Cinder is. Yay, a faster fork of CPython! For the love of God, please get those improvements to CPython. The world needs it!
> It's no secret how they made their runtimes fast.
So why is nobody doing it? The ecosystem that would benefit from it must be gigantic. Hell, I am a solo indie developer and I would happily pay $500 to see Python make a jump to match PHP's performance! If we extrapolate this to the whole Python ecosystem, the value of a fast CPython must be Billions.
> C extensions are already available in Python
What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language.
Giants spent their energy speeding things strategically:
・ Google wanted Chrome to win, so they spend their effort on JS.
・ Facebook was a PHP shop, so they worked on make it fast.
・ MS had to promote .net, so they optimized c#.
Now you also have to remember that:
・ it's a hard problem
・ python is not a performance oriented language, so it was not a priority
・ when data science became popular, c-extensions were usually enough to get the perfs you wanted
・ once you have a working python code reaching perfs limit, you could start a rewrite in a fast compiled language (Google did it with go, dropbox with rust, etc). But this could take years before you need it. Youtube ran fine on PHP then Python for almost a decade. Python can take you very, very far. In fact, I'd say half the time somebody says python is too slow for them, it's because they are doing something wrong. Case in point, I have a half-million users/day streaming website running on Python. It's been running for 10 years, and needs only one dev to maintain it.
・ there are actually a lot of improvements on tooling for perfs on python. Multiprocessing have fantastic pools and shared memory. We have asyncio (not to mention uvloop). c-extensions release the GIL, so you can use threads for parallelism with numpy. Most people are just not aware of it.
・ Python is very old, 4 years older than java. It has been design way before multi-core laptop were a thing. And so the GIL is everywhere.
・ python is in a very unique position: it's the second best language for anything. Meaning it's been used by data scientist, web devs, geographers, sysadmin, pen testers, os maintainers and biologists alike. They all have very different tooling, dev envs and workloads. Speed doesn't mean the same things for all of them. E.G: a slow starting JIT that speeds up a web server would destroy the use case for cli tools. Mercurial has not computational speed problems, but import time is killing it. But lazy imports would make debugging and new comers learning curve harder. What if they speed up things for them, but slow them down for you?
・ 2->3 made the entire community change averse. And to get speed, you need to break the C api somehow. Hence the Handle Python project.
So the previous attempts to speed up Python were by either interns, or people doing it out of passion/curiosity. A lot failed (unladen swallow, pyston1, etc). Some were successful but only for some dimension of speed and/or doing some compromises (pypy, stackless...). Some are still WIP (Pydgin, pyston 2..). The solutions that succeeded the most are actually side stepping the problem (cython, numpy, numba, dask) but usually became popular although some very good projects still deserve more light (nuitka, mypyc...)
It's only now that Python is becoming such a huge deal that MS hired Guido to work on general computational speed for cPython. Now that there is attention, money and dedication on the problem, you can already things moving in the bug tracker (coroutine unwrapping, c stack skipping calls, HandlePython, etc). It's going to take time. Millions have already been invested in V8 to make the JS slug go steroid, so catch up is not for tomorrow.
> What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language.
You can already do that with cython. And there are alternative for rust, nim, go...
> MS hired Guido to work on general computational speed for cPython.
van Rossum applied at Microsoft, they took him and he chose that project.
van Rossum had never shown any interest in JITs before. While at Google, he maintained that Python was fast enough. While at Dropbox, he initiated Asyncio (fixed and improved by many people) and integrated typing (Jukka Lehtosolo).
If a fast JIT emerges, it will likely be by existing Microsoft JIT experts. They should familiarize themselves with the cliquish nature of Python, rampant credit taking by those who did not do the work, and sidelining and active defamation of those outside the clique once the work is done.
It could be a major career disaster to get involved with these people.
> Facebook was a PHP shop, so they worked on make it fast.
Tangent: Facebook's work on PHP was largely a technical dead-end. Their work on HipHop and HHVM was never accepted upstream, and eventually resulted in their forking PHP into a separate and incompatible language called Hack, which basically nobody uses outside of Facebook.
In the meantime, the upstream PHP project hasn't been standing still. The core development team has made a number of significant improvements to PHP performance. By 2015 or so, PHP 7.0 matched the performance of HHVM, and newer versions have improved upon that. (PHP 8.0 added a JIT compiler, for example, which can dramatically boost performance for some applications.)
But can you swap out CPython for Cython in - say - a Django stack and things go smooth?
Every time I look up how to replace CPython with something faster for Django, everybody who tried it (in production) is of the opinion that it was surprisingly painful and is not worth the hassle.
I use Cython for the small part that has a tight loop in a Django project. Using Cython with Django itself sounds difficult, but the Falcon framework looks like it uses Cython.
Python is difficult to optimize due to complex language semantics. This excellent talk describes how the semantics of the "add" operation are much more complex than generally described. a+b is not just a.__add__(b), you have to read 200+ lines of CPython interpreter code to fully resolve its semantics: https://youtu.be/qCGofLIzX6g
Not saying it's impossible, but it's more of an uphill battle than optimizing other languages that have more regular semantics.
Python is also constrained by its C API, which is a very low-level and exposes a lot of implementation details to C extensions. Changing the API could improve performance but it would also make Python incompatible with a large ecosystem of C extensions.
I sometimes wish for Python to define a new C API that exposes far less, and to begin a long-term migration towards the new API so that eventually the interpreter would be less constrained. It would be a large migration, but unlike Python 2 -> 3 it wouldn't affect everyday users, only extension authors. I don't know if it will ever happen, but a person can dream.
26 comments
[ 4.3 ms ] story [ 72.2 ms ] threadWhat could it be?
Can CPython become faster? Could the devs take a look at how PHP and Javascript did it to make their runtimes so fast?
Can the Python language get extended so that you can define the bottlencks of your code in another language? Maybe in C? PHP? Javascript? Webassembly? Python bytecode?
Can a new runtime take over and everybody settle on it? Which one could it be?
> Can CPython become faster?
Yes. There are several people (including a team by Guido and Mark Shannon) working on making CPython faster in-tree. There are others, including the folks on Cinder and Pyjion, who are working on extensions to CPython out of tree.
> Could the devs take a look [...]
It's no secret how they made their runtimes fast. It's all -- more or less -- implementations of the original Smalltalk-80 paper and all that fell out of it.
> Can the Python language get extended so that you can define the bottlencks of your code in another language?
C extensions are already available in Python, and have been for quite some time. In fact, I would argue they are a little bit part of the problem: there are so many C extensions that runtime optimization efforts don't have as much effect as people would like; the slow stuff is already in C.
> Can a new runtime take over and everybody settle on it?
There are a lot of "competing" Python runtimes that serve different use-cases. Some use PyPy. Some use Jython. Some use Cinder. etc. Though everyone seems to ask "why not just use PyPy?", people forget to consider that everyone has different constraints that PyPy does not necessarily satisfy.
The Self language by David Ungar and others was an expansion of some of the SmallTalk ideas. If I remember correctly, Self is where polymorphic inline caching originated. Sun acqui-hired Urs Holzle (who did his PhD under Ungar), David Griswald, Gilad Bracha, and Lars Bak for their StrongTalk (statically typed dialect of SmallTalk) hybrid bytecode interpreter/JIT, which was the basis for the HotSpot JVM.
Lars Bak later created the original V8 JavaScript JIT for Chrome. When I was on the Google web search indexing team, I did a bit of work running a pre-release V8 in the indexing pipeline (to analyze text and formatting changes made to pages by JavaScript). If I remember correctly, at that time V8 wasn't a hybrid interpreter/JIT, but did quick-and-dirty generation of x86 machine code when a function was first invoked, and leaned heavily on polymorphic inline caching. My understanding is that early V8 was rather similar to the Self VM.
I believe ideas like tracing JITs originated a bit further from the SmallTalk ecosystem, but a lot of modern JIT ideas and techniques came from early SmallTalk and related research.
Only recently did I learn that Dartmouth BASIC did use a JIT approach as well.
It was only when dialects started to crop up on 8-bit computers that the trend for BASIC to be an interpreted language came to be.
[0] https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.97....
Smalltalk was one of the first JIT implementations,
"Efficient implementation of the smalltalk-80 system"
https://dl.acm.org/doi/10.1145/800017.800542
Then they followed up with StrongTalk,
https://strongtalk.org/
And SELF, which JavaScript OOP is modeled on,
https://selflanguage.org
You will recognize some of the V8 names on those papers.
Additionally, SELF JIT was repurposed as Hotspot by Java 1.2.
Although Java looks like C++ from syntax point of view, it actually is closer to Smalltalk and Objective-C in runtime semantics, thus it could profit a lot from that JIT research as well, even though it was originally targeted to dynamic languages.
And C extensions are an API, you don’t have to write them in C e.g. cython generates “C extensions”.
Furthermore there are other micro-hooks like numba ehich integrate rather easy and can provide great bang for your buck in restricted numerical contexts.
> It's no secret how they made their runtimes fast.
So why is nobody doing it? The ecosystem that would benefit from it must be gigantic. Hell, I am a solo indie developer and I would happily pay $500 to see Python make a jump to match PHP's performance! If we extrapolate this to the whole Python ecosystem, the value of a fast CPython must be Billions.
> C extensions are already available in Python
What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language.
Mostly, resources.
Giants spent their energy speeding things strategically:
・ Google wanted Chrome to win, so they spend their effort on JS.
・ Facebook was a PHP shop, so they worked on make it fast.
・ MS had to promote .net, so they optimized c#.
Now you also have to remember that:
・ it's a hard problem
・ python is not a performance oriented language, so it was not a priority
・ when data science became popular, c-extensions were usually enough to get the perfs you wanted
・ once you have a working python code reaching perfs limit, you could start a rewrite in a fast compiled language (Google did it with go, dropbox with rust, etc). But this could take years before you need it. Youtube ran fine on PHP then Python for almost a decade. Python can take you very, very far. In fact, I'd say half the time somebody says python is too slow for them, it's because they are doing something wrong. Case in point, I have a half-million users/day streaming website running on Python. It's been running for 10 years, and needs only one dev to maintain it.
・ the PSF had no money (https://pyfound.blogspot.com/2012/01/psf-grants-over-37000-t...), and most work on python was from people on their free time. Very few paid devs on Python for decades, compared to other languages
・ there are actually a lot of improvements on tooling for perfs on python. Multiprocessing have fantastic pools and shared memory. We have asyncio (not to mention uvloop). c-extensions release the GIL, so you can use threads for parallelism with numpy. Most people are just not aware of it.
・ Python is very old, 4 years older than java. It has been design way before multi-core laptop were a thing. And so the GIL is everywhere.
・ python is in a very unique position: it's the second best language for anything. Meaning it's been used by data scientist, web devs, geographers, sysadmin, pen testers, os maintainers and biologists alike. They all have very different tooling, dev envs and workloads. Speed doesn't mean the same things for all of them. E.G: a slow starting JIT that speeds up a web server would destroy the use case for cli tools. Mercurial has not computational speed problems, but import time is killing it. But lazy imports would make debugging and new comers learning curve harder. What if they speed up things for them, but slow them down for you?
・ 2->3 made the entire community change averse. And to get speed, you need to break the C api somehow. Hence the Handle Python project.
So the previous attempts to speed up Python were by either interns, or people doing it out of passion/curiosity. A lot failed (unladen swallow, pyston1, etc). Some were successful but only for some dimension of speed and/or doing some compromises (pypy, stackless...). Some are still WIP (Pydgin, pyston 2..). The solutions that succeeded the most are actually side stepping the problem (cython, numpy, numba, dask) but usually became popular although some very good projects still deserve more light (nuitka, mypyc...)
It's only now that Python is becoming such a huge deal that MS hired Guido to work on general computational speed for cPython. Now that there is attention, money and dedication on the problem, you can already things moving in the bug tracker (coroutine unwrapping, c stack skipping calls, HandlePython, etc). It's going to take time. Millions have already been invested in V8 to make the JS slug go steroid, so catch up is not for tomorrow.
> What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language.
You can already do that with cython. And there are alternative for rust, nim, go...
That...
van Rossum applied at Microsoft, they took him and he chose that project.
van Rossum had never shown any interest in JITs before. While at Google, he maintained that Python was fast enough. While at Dropbox, he initiated Asyncio (fixed and improved by many people) and integrated typing (Jukka Lehtosolo).
If a fast JIT emerges, it will likely be by existing Microsoft JIT experts. They should familiarize themselves with the cliquish nature of Python, rampant credit taking by those who did not do the work, and sidelining and active defamation of those outside the clique once the work is done.
It could be a major career disaster to get involved with these people.
I disagree, I think Python is usually a community where people gets the respect they are due.
Tangent: Facebook's work on PHP was largely a technical dead-end. Their work on HipHop and HHVM was never accepted upstream, and eventually resulted in their forking PHP into a separate and incompatible language called Hack, which basically nobody uses outside of Facebook.
In the meantime, the upstream PHP project hasn't been standing still. The core development team has made a number of significant improvements to PHP performance. By 2015 or so, PHP 7.0 matched the performance of HHVM, and newer versions have improved upon that. (PHP 8.0 added a JIT compiler, for example, which can dramatically boost performance for some applications.)
See Cython.
Have a look at cython – it provides exactly that.
Every time I look up how to replace CPython with something faster for Django, everybody who tried it (in production) is of the opinion that it was surprisingly painful and is not worth the hassle.
Not saying it's impossible, but it's more of an uphill battle than optimizing other languages that have more regular semantics.
Python is also constrained by its C API, which is a very low-level and exposes a lot of implementation details to C extensions. Changing the API could improve performance but it would also make Python incompatible with a large ecosystem of C extensions.
I sometimes wish for Python to define a new C API that exposes far less, and to begin a long-term migration towards the new API so that eventually the interpreter would be less constrained. It would be a large migration, but unlike Python 2 -> 3 it wouldn't affect everyday users, only extension authors. I don't know if it will ever happen, but a person can dream.
https://docs.micropython.org/en/v1.9.3/pyboard/reference/spe...
There are plenty of solutions for this, but having a standard that works cleanly across all kinds of python interpreters would be really good.
Maybe someone can make the viper code emitter into a PEP?