OS kernel runqueue is using a spinlock to schedule everything. So it works. Should you ever use a spinlock in application code? No. Let the OS via the synchronization primitives in whatever language your app is in.
Probably not, not without formal verification which is usually lacking.
Everyone's computers hang or get slow some of the time. Probably all of our locks have bugs in them, but good luck getting to the bottom of that, right now the industry is barely capable of picking a sorting algorithm that actually works.
It works if there is no scheduler, or you tell the scheduler what you're doing.
Turns out the first scenario is rare outside of embedded or OS development. The second scenario defeats the purpose because you're doing the same thing a mutex would be doing. It's not like mutexes were made slow on purpose to bully people. They're actually pretty fast.
The basic rule of writing your own cross-thread datastructures like mutexes or condition variables is... don't, unless you have very good reason not to. If you're in that rare circumstance where you know the library you're using isn't viable for some reason, then the next best rule is to use your OS's version of a futex as the atomic primitive, since it's going to solve most of the pitfalls for you automatically.
The only time I've manually written my own spin lock was when I had to coordinate between two different threads, one of which was running 16-bit code, so using any library was out of the question, and even relying on syscalls was sketchy because making sure the 16-bit code is in the right state to call a syscall itself is tricky. Although in this case, since I didn't need to care about things like fairness (only two threads are involved), the spinlock core ended up being simple:
As always: use standard libraries first, profile, then write your own if the data indicate that it's necessary. To your point, the standard library probably already uses the OS primitives under the hood, which themselves do a short userspace spin-wait and then fall back to a kernel wait queue on contention. If low latency is a priority, the latter might be unacceptable.
The following is an interesting talk where the author used a custom spinlock to significantly speed up a real-time physics solver.
I wrote my own spin lock library over a decade ago in order to learn about multi threading, concurrency, and how all this stuff works. I learned a lot!
Another somewhat known case of a spinlock is in trading, where for latency purposes the OS scheduler is essentially bypassed by core isolation and thread pinning, so there’s nothing better for the CPU to do than spinning.
"Unfair" paragraph is way too short. This is the main problem! The outlier starvation you get from contended spinlocks is extraordinary and, hypothetically, unbounded.
i always got the sense that spinlocks were about maximum portability and reliability in the face of unreliable event driven approaches. the dumb inefficient thing that makes the heads of the inexperienced explode, but actually just works and makes the world go 'round.
> Notice that in the Skylake Client microarchitecture the RDTSC instruction counts at the machine’s guaranteed P1 frequency independently of the current processor clock (see the INVARIANT TSC property), and therefore, when running in Intel® Turbo-Boost-enabled mode, the delay will remain constant, but the number of instructions that could have been executed will change.
rdtsc may execute out of order, so sometimes an lfence (previously cpuid) can be used and there is also rdtscp
Nice article! Yes, using spinlocks in normal userspace applications is not recommended.
One area where I found spinlocks to be useful is in multithreaded audio applications. Audio threads are not supposed to be preempted by other user space threads because otherwise they may not complete in time, leading to audio glitches. The threads have a very high priority (or have a special scheduling policy) and may be pinned to different CPU cores.
For example, multiple audio threads might read from the same sample buffer, whose content is occasionally modified. In that case, you could use a reader-writer-spinlock where multiple readers would be able to progress in parallel without blocking each other. Only a writer would block other threads.
What would be the potential problems in that scenario?
My concurrency knowledge is a bit rusty but aren't spinlocks only supposed to be used for very brief waits like in the hundreds of cycles (or situations where you can't block... like internal o/s scheduling structures in SMP setups)? If so how much does all this back off and starvation of higher priority threads even matter? If it is longer then you should use a locking primitive (except for in those low level os structures!) where most of the things discussed are not an issue. Would love to hear the use cases where spin locks are needed in eg user space, I dont doubt they occur.
what is/are the thread synchronization protocol called which is the equivalent to ethernet's CSMA? there's no "carrier sensing", but instead "who won or mistakes were made" sensing. or is that just considered a form of spinlock? (you're not waiting for a lock, you perform your operation then see if it worked; though you could make the operation be "acquire lock" in which case it's a spinlock)
I struggled with this in Wine. "malloc" type memory allocation involves at least two levels of spinlocks. When you do a "realloc", the spinlocks are held during the copying operation. If you use Vec .push in Rust, you do a lot of reallocs.
In a heavily multithreaded program, this can knock performance down by more than two orders of magnitude. It's hard to reproduce this with a simple program; it takes a lot of concurrency to hit futex congesion.
Real Windows, and Linux, don't have this problem. Only Wine's "malloc" in a DLL, which does.
Bug reports resulted in finger-pointing and denial.[1] "Unconfirmed", despite showing debugger output.
> The code is not thread-safe as, if multiple threads attempt to use this lock, we could read invalid values of isLocked (in theory, and on a CPU where tearing could happen on its word size).
The issue isn’t just tearing but also memory order. On some architectures you can read a valid but out of date value in Thread A after Thread B has updated that value. (Memory order is mentioned later in the article, to be fair.)
21 comments
[ 2.3 ms ] story [ 64.9 ms ] threadEveryone's computers hang or get slow some of the time. Probably all of our locks have bugs in them, but good luck getting to the bottom of that, right now the industry is barely capable of picking a sorting algorithm that actually works.
Turns out the first scenario is rare outside of embedded or OS development. The second scenario defeats the purpose because you're doing the same thing a mutex would be doing. It's not like mutexes were made slow on purpose to bully people. They're actually pretty fast.
The only time I've manually written my own spin lock was when I had to coordinate between two different threads, one of which was running 16-bit code, so using any library was out of the question, and even relying on syscalls was sketchy because making sure the 16-bit code is in the right state to call a syscall itself is tricky. Although in this case, since I didn't need to care about things like fairness (only two threads are involved), the spinlock core ended up being simple:
The following is an interesting talk where the author used a custom spinlock to significantly speed up a real-time physics solver.
Dennis Gustafsson – Parallelizing the physics solver – BSC 2025 https://www.youtube.com/watch?v=Kvsvd67XUKw
rdtsc may execute out of order, so sometimes an lfence (previously cpuid) can be used and there is also rdtscp
See https://github.com/torvalds/linux/blob/master/arch/x86/inclu...
And just because rdtsc is constant doesn't mean the processor clock will be constant that could be fluctuating.
The author should read https://webkit.org/blog/6161/locking-in-webkit/ so that they understand what they are talking about.
WebKit does it right in the sense that:
- It as an optimal amount of spinning
- Threads wait (instead of spinning) if the lock is not available immediately-ish
And we know that the algorithms are optimal based on rigorous experiments.
One area where I found spinlocks to be useful is in multithreaded audio applications. Audio threads are not supposed to be preempted by other user space threads because otherwise they may not complete in time, leading to audio glitches. The threads have a very high priority (or have a special scheduling policy) and may be pinned to different CPU cores.
For example, multiple audio threads might read from the same sample buffer, whose content is occasionally modified. In that case, you could use a reader-writer-spinlock where multiple readers would be able to progress in parallel without blocking each other. Only a writer would block other threads.
What would be the potential problems in that scenario?
Real Windows, and Linux, don't have this problem. Only Wine's "malloc" in a DLL, which does.
Bug reports resulted in finger-pointing and denial.[1] "Unconfirmed", despite showing debugger output.
[1] https://bugs.winehq.org/show_bug.cgi?id=54979
The issue isn’t just tearing but also memory order. On some architectures you can read a valid but out of date value in Thread A after Thread B has updated that value. (Memory order is mentioned later in the article, to be fair.)