> The notion of making the language more complex to be able to tell the compiler details like the above so that it would just do the RightThing(tm) is just crazy talk. Compilers are good at some things (mindless instruction scheduling, register allocation etc), but that shouldn't make you think they should be smart.
Compare this to Boehm 'Threads cannot be implemented as a library'.
The issue of course was with volatile which is useless, not with the notion of letting a compiler know about multi-threading assumptions.
I think today Linus has begrudgingly accepted that slowly moving the kernel to the C/C++11 acquire-release memory model, where possible, is a good thing.
Personally I had never fully grokked the reordering based memory barriers (I understood what they did, but it was hard for me to understand all the implications); on the other hand the C++11 memory model and its happens-before based somehow seems much more natural and it helps me retrospectively to understand even barrier based code.
Absolutely agree. I can't wrap my head around the store-load, load store etc. stuff but I think the Java memory model's happens before guarantees are quite digestible.
I think C++'s is based on that one, as Doug Lea worked on both. I should double check that factoid.
Oh, yes the C++ Memory model was definitely based on Java, although it has diverged a lot. FWIW, Doug is credited in a bunch of papers to the committee.
It doesn’t help that there are sometimes subtle but extremely important differences in the guarantees that certain types of barriers provide, be it at the hardware level, language level, VM level, etc.
Example: does a write with release semantics prevent writes to any address from being reordered past the barrier, or only writes to the same address? The answer for the general definition of “write-release semantics” depends on who you ask. The specifics for a given language, VM, or architecture may or may not be clearly stated. And even if they are, incomplete or wrong versions of the rules will no doubt spread across the Internet via the likes of Stack Overflow.
It's writes to any address. It would be quite useless otherwise, since it wouldn't then have the happens-before ordering with regards to all the plain (non-atomic) writes before that.
Also, reordering writes to the same address is uncommon to non-existent since the obvious meaning of that would break even single threaded semantics.
> It's writes to any address. It would be quite useless otherwise ...
Ugh, you're right, of course. My brain hasn't quite woken up today :-/. I remember there was some subtlety about read/acquire and write/release semantics that I consistently see different takes on.
I think it was actually whether write/release semantics prevent any memory operation or only other writes from being reordered, e.g., is it a StoreStore+StoreLoad barrier or just StoreStore? And, similarly, whether read/acquire semantics imply LoadLoad+LoadStore or just LoadLoad. My understanding is that write-release only guarantees a StoreStore barrier, and read-acquire only guarantees LoadLoad (and maybe LoadStore for direct dependencies?). I have, however, seen the stronger guarantees implied many times. The point is, regardless of which is 'correct', you are likely to stumble across an incorrect explanation from someone who ostensibly knows what they're talking about. And that itself is a problem.
I think the confusion stems from various languages and platforms providing stronger guarantees than are generally required to satisfy read/acquire and write/release. Java's `volatile`, I believe, requires both acquire+release semantics on writes. People get used to the stronger guarantees and then forget that language A's stronger guarantees aren't necessarily provided by language B.
It seems like there are roughly 4 levels of consistency
1. Sequential Consistency -- Total ordering exists for all atomics.
2. Acquire-Release Consistency -- Acquire barrier ensures all memory operations before the Acquire "happened before" the read of the spinlock. Release barrier ensures all memory operations before the release barrier "happens before" the write of the spinlock.
3. Consume-Release Consistency -- C++11 hypothetical: has not been used by C++ Compilers yet. This weaker memory ordering is a guarantee in some older chips but very few people seem to understand it.
4. Relaxed Consistency -- Atomics remain atomic, but no ordering is specified.
---------
It seems like #2 is sufficient for most cases, #1 is needed for a few obscure cases (the IRIW / Independent read-Independent write case). #3 is (probably) sufficient in cases where pointers are being used as the synchronization point, but very few programmers have studied consume-release and its straight up not-implemented in any major compiler yet. (Some CPUs do offer consume-release, which is why it is included in the C++11 standard).
#4 is rare, but useful if anyone wants the absolute fastest and knows atomics are enough (ex: a global multithreaded counter)
> Personally I had never fully grokked the reordering based memory barriers (I understood what they did, but it was hard for me to understand all the implications); on the other hand the C++11 memory model and its happens-before based somehow seems much more natural and it helps me retrospectively to understand even barrier based code.
Same here. C++11 memory model is when I managed to "wake up" to the facts behind barriers, load/load and store/store dependencies, and all that good stuff.
I recognize that Java apparently specified all of this stuff first, but I wasn't a Java programmer at that time. So I learned it from C++11 (Or more specifically: Herb Sutter's talks + other C++11 programmers)
---------
EDIT: For anybody who has issues with memory barriers, think about a spinlock and where the appropriate memory barriers need to go for a spinlock to work. That's the real beauty of C++11/Java model: instead of thinking in terms of load/load or load/store or store/store barriers, you think in terms of "lock acquire" and "unlock release".
The spinlock is the canonical example of memory barriers, and "remembering" that through the use of language is helpful.
There's so much there its hard to pick out anything in particular in Preshing's blog as a "must read". Its all very useful, but its also spread out over a lot of posts. I just picked the first link from search engines
The C++11 draft standard defines "happens before" and all that good stuff, but it is a 1000+ page document. OpenCL 2.0 copy/pastes from the C11 / C++11 standards and puts it all in one section: 3.3.6, which is only about 3ish pages long (+additional pages for OpenCL specific stuff).
OpenCL2.0 is missing consume/release semantics, but nobody implemented that yet anyway (even in C++ world, no one has bothered to properly implement a consume/release compiler yet).
I guess OpenCL2.0 has the whole "local vs global" memory order thing going on, while C11 / C++11 is all one memory space. But I personally found OpenCL2.0's specification to be easier to read than the C++11 draft standard.
11 comments
[ 5.7 ms ] story [ 38.0 ms ] threadCompare this to Boehm 'Threads cannot be implemented as a library'.
The issue of course was with volatile which is useless, not with the notion of letting a compiler know about multi-threading assumptions.
I think today Linus has begrudgingly accepted that slowly moving the kernel to the C/C++11 acquire-release memory model, where possible, is a good thing.
Personally I had never fully grokked the reordering based memory barriers (I understood what they did, but it was hard for me to understand all the implications); on the other hand the C++11 memory model and its happens-before based somehow seems much more natural and it helps me retrospectively to understand even barrier based code.
I think C++'s is based on that one, as Doug Lea worked on both. I should double check that factoid.
Example: does a write with release semantics prevent writes to any address from being reordered past the barrier, or only writes to the same address? The answer for the general definition of “write-release semantics” depends on who you ask. The specifics for a given language, VM, or architecture may or may not be clearly stated. And even if they are, incomplete or wrong versions of the rules will no doubt spread across the Internet via the likes of Stack Overflow.
Also, reordering writes to the same address is uncommon to non-existent since the obvious meaning of that would break even single threaded semantics.
Ugh, you're right, of course. My brain hasn't quite woken up today :-/. I remember there was some subtlety about read/acquire and write/release semantics that I consistently see different takes on.
I think it was actually whether write/release semantics prevent any memory operation or only other writes from being reordered, e.g., is it a StoreStore+StoreLoad barrier or just StoreStore? And, similarly, whether read/acquire semantics imply LoadLoad+LoadStore or just LoadLoad. My understanding is that write-release only guarantees a StoreStore barrier, and read-acquire only guarantees LoadLoad (and maybe LoadStore for direct dependencies?). I have, however, seen the stronger guarantees implied many times. The point is, regardless of which is 'correct', you are likely to stumble across an incorrect explanation from someone who ostensibly knows what they're talking about. And that itself is a problem.
I think the confusion stems from various languages and platforms providing stronger guarantees than are generally required to satisfy read/acquire and write/release. Java's `volatile`, I believe, requires both acquire+release semantics on writes. People get used to the stronger guarantees and then forget that language A's stronger guarantees aren't necessarily provided by language B.
It seems like there are roughly 4 levels of consistency
1. Sequential Consistency -- Total ordering exists for all atomics.
2. Acquire-Release Consistency -- Acquire barrier ensures all memory operations before the Acquire "happened before" the read of the spinlock. Release barrier ensures all memory operations before the release barrier "happens before" the write of the spinlock.
3. Consume-Release Consistency -- C++11 hypothetical: has not been used by C++ Compilers yet. This weaker memory ordering is a guarantee in some older chips but very few people seem to understand it.
4. Relaxed Consistency -- Atomics remain atomic, but no ordering is specified.
---------
It seems like #2 is sufficient for most cases, #1 is needed for a few obscure cases (the IRIW / Independent read-Independent write case). #3 is (probably) sufficient in cases where pointers are being used as the synchronization point, but very few programmers have studied consume-release and its straight up not-implemented in any major compiler yet. (Some CPUs do offer consume-release, which is why it is included in the C++11 standard).
#4 is rare, but useful if anyone wants the absolute fastest and knows atomics are enough (ex: a global multithreaded counter)
Same here. C++11 memory model is when I managed to "wake up" to the facts behind barriers, load/load and store/store dependencies, and all that good stuff.
I recognize that Java apparently specified all of this stuff first, but I wasn't a Java programmer at that time. So I learned it from C++11 (Or more specifically: Herb Sutter's talks + other C++11 programmers)
---------
EDIT: For anybody who has issues with memory barriers, think about a spinlock and where the appropriate memory barriers need to go for a spinlock to work. That's the real beauty of C++11/Java model: instead of thinking in terms of load/load or load/store or store/store barriers, you think in terms of "lock acquire" and "unlock release".
The spinlock is the canonical example of memory barriers, and "remembering" that through the use of language is helpful.
* https://www.youtube.com/watch?v=lVBvHbJsg5Y
EDIT: Honorable mention is Preshing's blog. https://preshing.com/20131125/acquire-and-release-fences-don...
There's so much there its hard to pick out anything in particular in Preshing's blog as a "must read". Its all very useful, but its also spread out over a lot of posts. I just picked the first link from search engines
EDIT2: Honorable mention#2: The OpenCL 2.0 standard section 3.3.6: https://www.khronos.org/registry/OpenCL/specs/opencl-2.0.pdf
The C++11 draft standard defines "happens before" and all that good stuff, but it is a 1000+ page document. OpenCL 2.0 copy/pastes from the C11 / C++11 standards and puts it all in one section: 3.3.6, which is only about 3ish pages long (+additional pages for OpenCL specific stuff).
OpenCL2.0 is missing consume/release semantics, but nobody implemented that yet anyway (even in C++ world, no one has bothered to properly implement a consume/release compiler yet).
I guess OpenCL2.0 has the whole "local vs global" memory order thing going on, while C11 / C++11 is all one memory space. But I personally found OpenCL2.0's specification to be easier to read than the C++11 draft standard.