My experience has been the opposite. If lean had linear types (or separation types), it would be, but as it is, Lean's just a little bit too focused on talking about results to tidily talk about how those results are computed.
I don't know. For many operations, you can encode "how" by saying "under any permutation of this sequence of applications". At least, for EREW machines.
As I found out recently, there's a lighter option: model checkers like Spin. You describe your synchronization logic in a small modeling language (Promela), and Spin tries every possible interleaving of that model.
That's a useful interpretation of the two terms, but it's far from universal and the two have often been used fairly interchangeably over the decades. It's been much more useful as a distinction when someone discussing it announces that that is how they're separating the two concepts, instead of trying to force other people to adopt that particular pair of definitions.
Frankly the software industry suffers heavily from a lack in standardized terminology. The precise definition of parallelism vs concurrency is one that I think is incredibly important. You are doing your peers a disservice by using them interchangeably, they are not.
That battle has unfortunately been lost and different sources give different definitions, often exactly swapped. This was discussed in one of the HN posts linked in the article: https://news.ycombinator.com/item?id=36318280
In the end I don't think it is too much of an issue. What confusion is really brought by conflating parallelism and concurrency? Sure, concurrent programs can be serialized onto a single core. But there isn't some deep conceptual unlock you get by having a strict conceptual boundary between concurrency and parallelism.
I think there is a deep conceptual unlock: concurrency is about semantics, whilst parallelism is an operational property. I use this distinction a lot in my own work. Concurrent programming primitives are inherently non-deterministic (and usually about handling non-deterministic events), on top of which we must then establish some kind of properties (sometimes determinism to some extent). Many interesting parallel operations are however completely deterministic, and the fact that they are parallel is a property of their assigned cost model (and hopefully implementation, in practice).
I agree that this distinction is hardly universal, but it seems to be growing increasingly established, and I think it is worth fighting for it.
I don't like the essential characteristic of concurrency being nondeterminism. its really that multiple processes are running concurrently. if we don't have serializing operations, we have arbitrary execution order. but if we do then we can introduce the necessary determinism while still being (largely) concurrent in evaluation. and if those logically concurrent processes are physically concurrent then we have parallelism. so the first is necessary but not sufficient for the latter.
so I find saying that we have one or the other to pretty misleading.
Personally I think it's not a good idea to think too rigidly about and try to draw a huge distinction between the two. They're on a continuum and sometimes I'd say some things aren't even strictly speaking "between" them either. Sitting down and trying to classify code into "parallel" and "concurrent" is as likely to do harm as to do any good.
I firmly disagree, they are not a continuum, they are binary properties of what they describe. Code can have concurrency primitives, but parallelism primitives must necessarily come from the environment the code executes in, whether it's multiple code streams on a multicore processor or process parallelism provided by an operating system. Programs that are parallel are necessarily also concurrent (if they must communicate between parallel executions), but the inverse is not necessarily true.
If this distinction wasn't important, Python's infamous GIL would not be an issue.
As other comments have pointed out, this is just not true in general usage.
When I was a grad student studying this stuff (~20 years ago), we used "parallelism" to mean running on different cores at the same time and "concurrency" to mean preemptive multithreading on a single processor.
Parallelism without concurrency is useless, and concurrency without parallelism is usually cooperative and does not have the same challenges. So in essence, the title is correct.
I would probably recommend the art of multiprocessor programming (herlihy and shavitz) as a good starting point for concurrent programming. There is also " A primer on memory consistency and cache coherence" (Nagarajan et al) if you want to get more into the interaction between memory consistency and coherence.
The Raku language (formerly Perl 6) and its underlying VM has features to support parallel programming, concurrency and asynchrony, designed to make common cases relatively easy to code and avoid pitfalls.
Jonathan Worthington, the author of the VM and these features, has given an excellent presentation on the concepts and their implementation.
What kind of tooling were you using? IMO, deadlocks are some of the easiest concurrency bugs to diagnose. If you can see the thread stacks, it is easy to see threads are blocked from acquiring a lock. If you can attach a debugger, it is easy to see which locks are involved. Then you can pretty much figure things out using the straightforward guideline that if multiple locks are involved, they must be acquired in the same order in all code paths.
I don't want to devalue your experience, but I am surprised to hear that. Livelock is harder to debug. Silent data corruption caused by missing or wrong synchronization is way harder to debug.
> Silent data corruption caused by missing or wrong synchronization is way harder to debug.
Reminds me of being a young and ambitious C++ programmer 25 years ago, discovering that when you have a map and do “return m[k]”, it is not, in fact, a read-only operation when k does not exist. After which I learned that const-correctness is not just a nice-to-have, especially in multithreaded applications.
But yeah deadlocks are hardly ever a difficult issue to diagnose. They may potentially be difficult to resolve, but at that point, it very much suggests that there’s an architecture / design issue.
43 comments
[ 0.24 ms ] story [ 3.3 ms ] threadYou can't unit test your way out, but if you care about the code's correctness, today there's a way.
As I understand it, the parallelism is about task execution and concurrency is about task structure. Or, as Rob Pike said:
"Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once."
He said that in his *Concurrency is not Parallelism" talk.
In the end I don't think it is too much of an issue. What confusion is really brought by conflating parallelism and concurrency? Sure, concurrent programs can be serialized onto a single core. But there isn't some deep conceptual unlock you get by having a strict conceptual boundary between concurrency and parallelism.
I agree that this distinction is hardly universal, but it seems to be growing increasingly established, and I think it is worth fighting for it.
so I find saying that we have one or the other to pretty misleading.
If this distinction wasn't important, Python's infamous GIL would not be an issue.
When I was a grad student studying this stuff (~20 years ago), we used "parallelism" to mean running on different cores at the same time and "concurrency" to mean preemptive multithreading on a single processor.
But that's the only nit
not not
7 1 5 6 9 0
Ask Claude, which has read all the existing literature on parallel programming, to make the program faster.
Jonathan Worthington, the author of the VM and these features, has given an excellent presentation on the concepts and their implementation.
https://www.youtube.com/watch?v=JpqnNCx7wVY
I don't want to devalue your experience, but I am surprised to hear that. Livelock is harder to debug. Silent data corruption caused by missing or wrong synchronization is way harder to debug.
Reminds me of being a young and ambitious C++ programmer 25 years ago, discovering that when you have a map and do “return m[k]”, it is not, in fact, a read-only operation when k does not exist. After which I learned that const-correctness is not just a nice-to-have, especially in multithreaded applications.
But yeah deadlocks are hardly ever a difficult issue to diagnose. They may potentially be difficult to resolve, but at that point, it very much suggests that there’s an architecture / design issue.
Much like we get pilots comfortable in single engine aircraft before we have them fly around in 747s and AC130s.