As someone who has done a lot of atomics programming: stick with sequential consistency unless you are using the variables in a very well known pattern (i.e. the double locking pattern, ring buffers, etc).
Additionally, atomic doesn't mean lock free. But if they are lock free [0] you can use atomics in shared memory across a process barrier which is a lot of fun. Another good resource on this is: https://preshing.com/20120612/an-introduction-to-lock-free-p...
Sequential consistency is always "correct" in that you can substitute it for the others and your program will be correct -- but it's generally not warranted in my experience, to the point that I find liberal use of it to be a code smell, indicative that someone may have just sprinkled them around and "hoped for the best" rather than analyzing ordering requirements of their algorithm.
> stick with sequential consistency unless you are using the variables in a very well known pattern
This has been popular advice, and is presumably why WG21 chose to default the ordering to sequentially consistent when standardising these types. But, if you look closer it doesn't necessarily make sense.
Sequential Consistency is very expensive. Every execution context must ensure that it sees everything marked Sequentially Consistent in the same order, no matter what that order is, and so your compiler and then CPU is likely going to treat this very pessimistically. In contrast there's a good chance your high-end CPU in a server or personal computer can do Acquire-Release nice and fast if your data layout is friendly, and almost anything that's not a toy can do Relaxed [edited to correct] very fast indeed because that's easy.
So if you could correctly have used Acquire-Release, but you picked (or in C++ allowed the default) Sequentially Consistent because you were scared, you are leaving orders of magnitude of performance on the table.
At the other side, Sequential Consistency doesn't ensure correctness. If your algorithm wouldn't deliver what you intended under any memory ordering, Sequential Consistency can't re-order things so that it works - it's still busted although I guess debugging it might be easier now since all your execution contexts agree about the order in which things went wrong.
I think the better advice is probably something like, "If you're not sure, Sequentially Consistent isn't what you wanted, what you wanted is to not use atomic memory ordering".
From my understanding, SeqCst on x86 is free since the underlying CPU has a strong memory model anyway. Only on other platforms do you get the performance benefits.
That is incorrect but a common misconception (I used to believe the same). Try using it and you'll see the generated code is slower for stores on x86: https://godbolt.org/z/KrzT9bTKf
I agree with GP - the commonly given "always use sequentially consistent" advice is not really good advice. It should be "use a mutex wherever possible", but once you decide that's not performant enough, you probably often do want acq/rel or relaxed. I've actually found seq_cst to be quite rare.
In addition to what dataflow already said: while something might look free because you don't need to emit any kind of special instruction to get the desired result, doesn't mean that it's free in hardware. The reason why a lot of CPU architectures have weaker memory ordering is because this is faster on a hardware level. Consider that for sequential consistency to work on a multithreaded system, caches of all cores must be kept in sync. This is hidden from you, but does result in higher latencies and more power usage.
> The C++ standard recommends (but does not require) that lock-free atomic operations are also address-free, that is, suitable for communication between processes using shared memory.
So you must check your compiler's documentation before using lock-free atomics in shared memory across a process barrier.
On one hand, the author says that instanciating the singleton and storing it is not safe because there is no synchronization, but on the other hand he says that null check before is safe because the mutex introduces synchronization.
My understanding would be that the mutex protects this whole section and that the atomic is not necessary at all in this case.
Consider the code without the initial null check optimization. Everything occurs under the mutex in this case, so no atomics are needed -- it's fully serialized. (That is, all memory operations which occur within the mutex will be visible to any other thread which subsequently acquires the mutex.)
Now add in the initial null check. Because it occurs outside the mutex, it has no ordering established with anything that occurs under the mutex. Beside that p itself must be atomic to avoid a data race with itself (torn read), you need some way to ensure that App is visible if p was found to be set. Since the mutex isn't touched here, it doesn't provide any help. You need to establish that relationship using p itself. Hence the additional release-acquire sequence on p.
Btw a simple mental model for mutexes is that of the spin lock, acquired using an acquire-release operation (e.g. test-and-set), released using a release operation (e.g. write). There's no other "magic" that happens in a mutex to make them special memory-order wise. E.g. mutexes (nor memory ordering generally) do not "force" things out to memory or such.
tl;dr: Remember that the "new" operator does two operations: allocates memory and fills in the memory with the data. Now, if you have two threads (A & B):
- A allocates memory and then gets pre-empted
- B will pass the null pointer check and attempt to use the non-filled in memory block
This website consists of content copied and translated from other Chinese websites (possibly without permission). The homepage is pumping out articles much faster than I would expect a person to be able to write them (though that's subjective). Additionally this article is a machine or human-translated version of https://luyuhuang.tech/2022/06/25/cpp-memory-order.html (both articles have a Chinese comment of `将变量 a 加载到寄存器`, which was not translated into English; I would expect an author to not put Chinese comments into English articles) but the source article goes completely unattributed. The adjacent article https://www.sobyte.net/post/2022-06/shell-1/ is a translated version of https://a-wing.top/shell/2021/05/01/sh-compatibles-history, which copies the code comments verbatim while also not citing the source.
The website's About page says, "What I’ve read, what I’ve learned." So I guess they're technically telling users the articles aren't actually written by the blog's owner? Of course the blog owner includes Google Ads on articles taken from other sites without attribution.
At least the recent articles at least have the courtesy to link to their original sources. The latest article from 9 days ago (https://www.sobyte.net/post/2023-08/gonew/) is a translated version of https://tonybai.com/2023/08/11/introduction-to-the-gonew-too... (from a different author) and shares the same snippets, though this article includes the original article's URL at the bottom (in a non-clickable code block to avoid sharing SEO reputation I guess?).
EDIT: The stolen content was actually noticed last time sobyte.net showed up on HN (https://news.ycombinator.com/item?id=32456336), but that user did not investigate to find that multiple articles were all translated without attribution.
Articles like these would be a lot shorter if Alpha had never existed. A significant chunk of this complexity is attributable to Alpha and its goofball ability to reorder loads and stores in ways that programmers never wanted. There are details of the C++ memory model that are impossible to observe on any hardware that exists today.
My contentious view on this is that almost all of the C++ memory model is best ignored.
If you write using acquire-fence, release-fence, and everything else relaxed order using gcc style intrinsics then reasoning about the system becomes much easier. As far as I can tell you've also written the optimally performing code. That is, ignore the atomic type entirely and most of the operations.
The qualifier above is that it's very difficult to determine what the order annotation on an operation actually means. A fence is certain, it affects everything. An acquire-release fetch_add? Either it only affects ordering of other accesses to the same address or it works as if there were appropriately positioned fences beside it, and I'm not confident there is consensus on which interpretation is correct.
If your system considers the order annotation to only refer to that particular address, then there are patterns that can be written with lower overhead than the pure fences strategy. I don't think hardware or compilers work on that basis.
Sequentially consistent ordering on parallel systems is also the wrong thing. If you want a magic clock of serialised execution, why are you bothering with atomics.
17 comments
[ 4.0 ms ] story [ 60.4 ms ] threadAdditionally, atomic doesn't mean lock free. But if they are lock free [0] you can use atomics in shared memory across a process barrier which is a lot of fun. Another good resource on this is: https://preshing.com/20120612/an-introduction-to-lock-free-p...
[0] https://en.cppreference.com/w/cpp/atomic/atomic/is_lock_free
This has been popular advice, and is presumably why WG21 chose to default the ordering to sequentially consistent when standardising these types. But, if you look closer it doesn't necessarily make sense.
Sequential Consistency is very expensive. Every execution context must ensure that it sees everything marked Sequentially Consistent in the same order, no matter what that order is, and so your compiler and then CPU is likely going to treat this very pessimistically. In contrast there's a good chance your high-end CPU in a server or personal computer can do Acquire-Release nice and fast if your data layout is friendly, and almost anything that's not a toy can do Relaxed [edited to correct] very fast indeed because that's easy.
So if you could correctly have used Acquire-Release, but you picked (or in C++ allowed the default) Sequentially Consistent because you were scared, you are leaving orders of magnitude of performance on the table.
At the other side, Sequential Consistency doesn't ensure correctness. If your algorithm wouldn't deliver what you intended under any memory ordering, Sequential Consistency can't re-order things so that it works - it's still busted although I guess debugging it might be easier now since all your execution contexts agree about the order in which things went wrong.
I think the better advice is probably something like, "If you're not sure, Sequentially Consistent isn't what you wanted, what you wanted is to not use atomic memory ordering".
I agree with GP - the commonly given "always use sequentially consistent" advice is not really good advice. It should be "use a mutex wherever possible", but once you decide that's not performant enough, you probably often do want acq/rel or relaxed. I've actually found seq_cst to be quite rare.
> The C++ standard recommends (but does not require) that lock-free atomic operations are also address-free, that is, suitable for communication between processes using shared memory.
So you must check your compiler's documentation before using lock-free atomics in shared memory across a process barrier.
On one hand, the author says that instanciating the singleton and storing it is not safe because there is no synchronization, but on the other hand he says that null check before is safe because the mutex introduces synchronization.
My understanding would be that the mutex protects this whole section and that the atomic is not necessary at all in this case.
Please correct me if I'm wrong.
Consider the code without the initial null check optimization. Everything occurs under the mutex in this case, so no atomics are needed -- it's fully serialized. (That is, all memory operations which occur within the mutex will be visible to any other thread which subsequently acquires the mutex.)
Now add in the initial null check. Because it occurs outside the mutex, it has no ordering established with anything that occurs under the mutex. Beside that p itself must be atomic to avoid a data race with itself (torn read), you need some way to ensure that App is visible if p was found to be set. Since the mutex isn't touched here, it doesn't provide any help. You need to establish that relationship using p itself. Hence the additional release-acquire sequence on p.
Btw a simple mental model for mutexes is that of the spin lock, acquired using an acquire-release operation (e.g. test-and-set), released using a release operation (e.g. write). There's no other "magic" that happens in a mutex to make them special memory-order wise. E.g. mutexes (nor memory ordering generally) do not "force" things out to memory or such.
tl;dr: Remember that the "new" operator does two operations: allocates memory and fills in the memory with the data. Now, if you have two threads (A & B):
- A allocates memory and then gets pre-empted
- B will pass the null pointer check and attempt to use the non-filled in memory block
That's the part I missed : the first null check can be skipped if the memory is allocated but the constructor hasn't been executed.
The website's About page says, "What I’ve read, what I’ve learned." So I guess they're technically telling users the articles aren't actually written by the blog's owner? Of course the blog owner includes Google Ads on articles taken from other sites without attribution.
At least the recent articles at least have the courtesy to link to their original sources. The latest article from 9 days ago (https://www.sobyte.net/post/2023-08/gonew/) is a translated version of https://tonybai.com/2023/08/11/introduction-to-the-gonew-too... (from a different author) and shares the same snippets, though this article includes the original article's URL at the bottom (in a non-clickable code block to avoid sharing SEO reputation I guess?).
EDIT: The stolen content was actually noticed last time sobyte.net showed up on HN (https://news.ycombinator.com/item?id=32456336), but that user did not investigate to find that multiple articles were all translated without attribution.
But we already had this discussion I think.
If you write using acquire-fence, release-fence, and everything else relaxed order using gcc style intrinsics then reasoning about the system becomes much easier. As far as I can tell you've also written the optimally performing code. That is, ignore the atomic type entirely and most of the operations.
The qualifier above is that it's very difficult to determine what the order annotation on an operation actually means. A fence is certain, it affects everything. An acquire-release fetch_add? Either it only affects ordering of other accesses to the same address or it works as if there were appropriately positioned fences beside it, and I'm not confident there is consensus on which interpretation is correct.
If your system considers the order annotation to only refer to that particular address, then there are patterns that can be written with lower overhead than the pure fences strategy. I don't think hardware or compilers work on that basis.
Sequentially consistent ordering on parallel systems is also the wrong thing. If you want a magic clock of serialised execution, why are you bothering with atomics.