This is well-written. I could follow along quite nicely, from the setup through the bottlenecks and onto the resolution of the performance bug. Even the PRs are very pleasant to read: the majority of them is just a handful of changed lines with an added tests and a bit of documentation.
I was taken aback for a moment that this work originated from a report on StackOverflow. I had thought SO was effectively dead and abandoned by its community. But maybe I shouldn't project my own experience onto everyone else.
I’m not sure why it took me, a NumPy developer, looking at the benchmark numbers and saying “hmm, this is a bug”. But that is what it took. People are sometimes slow to treat behavior in dependencies like NumPy as bugs.
or maybe SO is back to its community sans clout chasers and tourists? If a vaninishing minority drive a given community in content generation and discourse, then lurkers etc leaving isnt as meaningful. you could lose 99% of users on many platforms without disrupting the core community and often having the added benefit of imoroving SNR.
It's a less marketable product in the modern attention economy, but that isnt the same as a dead community. It would be interesting to see a plot of posts/discussion liveliness per unit time and seeing is quality and depth of both questions and answers has changed and how they have changed.
> only acquire the lock when the flag needs to be updated
Unclear why you still need the lock here in that case. The idea that this flag may get updated during runtime and impacts how the software works when set seems to clash with the idea we need take no action having performed a relaxed (ie non-synchronising) load and seen it wasn't set at some previous time.
Maybe there's something I don't understand about these internals, which may be as simple as "It's just advisory so if we don't trace when we should no big deal".
I know this is more or less expected, but the improvement induced by adding a worker diminishes very rapidly... I guess it's not the cpython/numpy's fault but rather the CPU.
I thought NumPy was already releasing the GIL. On regular non-free-threaded Python, you can run threaded parallel Numpy operations and have multiple cores doing 100%, I've relied on that. Maybe not the case with the operations this article focuses on (sin/cos).
Yes, numpy does release the GIL. But the code in question has multiple numpy calls, called in a loop:
sum((np.sin(np.cos(np.sin(np.cos(x + i)))).sum()
for i in range(n_loop)))
This is not like:
release GIL
# i = 0
compute y = x + 0 (numpy broadcasting sum)
compute z = np.cos(y) (elementwise)
...
add to running total
# i = 1
compute y = x + 1
...
reacquire GIL
Instead it is:
# i = 0
look up "+" operation
release GIL
compute y = x + 0
reacquire GIL
look up "np.cos" operation
release GIL
compute z = np.cos(y)
reacquire GIL
...
# i = 1
look up "+" operation
release GIL
compute y = x + 1
reacquire GIL
...
So there was work being protected by the GIL, that suddenly is exposed to lock contention with free threading.
Of course, without free threading, the lock contention would be way worse, but this time the GIL is the lock being contended. Numpy has to reacquire the GIL whenever it returns from a function call, and this expression is made up of multiple calls. To multithread effectively with numpy (in non-freethreading) you'd normally aim to vectorise into a small number of calls in big arrays.
The composition of +, then np.cos, etc. is not too bad if these are big arrays, but the problem is the pure Python iteration over the range which is, presumably, quite large. You could vectorise over the range:
12 comments
[ 146 ms ] story [ 920 ms ] threadI was taken aback for a moment that this work originated from a report on StackOverflow. I had thought SO was effectively dead and abandoned by its community. But maybe I shouldn't project my own experience onto everyone else.
It's a less marketable product in the modern attention economy, but that isnt the same as a dead community. It would be interesting to see a plot of posts/discussion liveliness per unit time and seeing is quality and depth of both questions and answers has changed and how they have changed.
Unclear why you still need the lock here in that case. The idea that this flag may get updated during runtime and impacts how the software works when set seems to clash with the idea we need take no action having performed a relaxed (ie non-synchronising) load and seen it wasn't set at some previous time.
Maybe there's something I don't understand about these internals, which may be as simple as "It's just advisory so if we don't trace when we should no big deal".
Of course, without free threading, the lock contention would be way worse, but this time the GIL is the lock being contended. Numpy has to reacquire the GIL whenever it returns from a function call, and this expression is made up of multiple calls. To multithread effectively with numpy (in non-freethreading) you'd normally aim to vectorise into a small number of calls in big arrays.
The composition of +, then np.cos, etc. is not too bad if these are big arrays, but the problem is the pure Python iteration over the range which is, presumably, quite large. You could vectorise over the range:
but this is the start of a new conversation.