That was largely a semantic issue, which I responded with:
Parallelism: simultaneous calculations executing in the service of a single problem, usually with the goal of improved performance.
Concurrency: executions in the same time granularity, but not necessarily simultaneous. Also not necessarily in the service of the same problem, but some form of synchronization is required.
What I will add this time is that there are different kinds of parallelism, and I think he's considering "data parallelism" the real parallelism. I think this is a false distinction. What he calls "concurrency," I and others call either task parallelism (for distinct tasks that don't need to communicate much) or pipeline parallelism (for tasks that work on data in stages).
Yep, you are right that I am missing out on the fact that there is different types of parallelism in the post. But it was getting too big of a post already :/
You can also classify parallelism. An example is to compare that you annotate parallel execution on the data structure (this is a parallel array) or on the control-flow (when operating on this array, do it in parallel). Another is if the parallel distinction is largely dynamic (worker pool) or static (we know we have 16 cores so the optimal communication structure assuming a double torus pairing configuration is ...). Yet another is to look at the data themselves. Are they a flat array or a nested tree structure (Nested Data Level parallelism is cool btw). And finally, to stop all the examples, we could look at declaratively making the code parallel and doing so explicitly yourself from primitives (think rolling your own database versus writing queries in SQL for an existing one - and hope the query optimizer can help).
The main point is that Erlang has taken a choice here. And that choice only covers some uses of what people may want to do. For the other things, people tend to use Erlang as a controller and then write their FPGA code and run it separately. This is what the original telephony switches were doing with Erlang.
Parallelism is an algorithms problem: given n processors, design an algorithm that runs faster as n increases.
Concurrency is a systems problem: design a system that safely runs n separate but interacting contexts of execution. Erlang's solution to concurrency--no communication between processes other than asymmetric message passing, no shared state other than the process registry--is a great solution to concurrency in terms of safety and distributon across multiple hosts, but it provides limited help to parallelism.
Erlang's reputation as a great language for concurrent programming on multicore processors is largely by accident--it has a great concurrency model for networks of distributed machines, and almost any concurrency model is better than mutexes and threads, but on multicore machines, Erlang wins only by default.
no communication between processes other than asymmetric message passing, no shared state other than the process registry--is a great solution to concurrency in terms of safety and distributon across multiple hosts, but it provides limited help to parallelism.
That's the same model of executions that MPI supports, and the most high-performance, parallel programs ever written have been written with MPI - scaling to thousands of processors across hundreds of nodes.
I feel that the people who are trying to define "parallelism" and "concurrency" are clashing with existing definitions from the already established high-performance computing community.
A thorough read would mention the semantic difference between them, the granularity of parallelism, some of the intent behind the concurrency/parallelism constructs, implications of performance when trying to do parallelism in Erlang, general parallel execution theory and strategies related to it.
Concurrency - property of systems in which several computational processes are executing at the same time, and potentially interacting with each other.
Parallelism - computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel").
Before introduction of SMP VM - Erlang was strictly concurrent, with only ability to write parallelism via distribution (i.e. multi-node systems). With SMP VM you can write single-node parallel code too.
Not, that it's the best language for HPC, but you can always call C/C++, Fortran, CUDA/OpenCL from it using NIFs.
I've read several authoritative books on Erlang and all of them used the term "concurrency" not "parallelism." I'm not sure who's running around saying Erlang provides parallelism.
I'm dubious that Haskell is better for writing distributed concurrent systems, but I don't know much about haskell.
Rather than bashing erlang like this, I'd have gotten a lot more out of an article that described how Haskell approached the problem, possibly with contrasts to erlang.
But when you start off with at best nonsense "erlang's parallelism is not parallelism" you're going to lose people who do know something about erlang.
Here's the deal with erlang: The syntax throws some people off. If you will spend a couple hours learning the language (maybe a couple days) you'll start to appreciate the syntax and it will no longer be a problem.
Erlang is not as fast as bare metal languages like, say, C, but Erlang is fantastic for making programs in parallel and executing them on multiple cores or nodes or clusters of nodes with multiple cores.
I guess I should take the occurrence of these kinds of articles as a sign that Erlang is getting wider acceptance. That's great!
But please, cheer your language, don't bash the competition.
"I'm dubious that Haskell is better for writing distributed concurrent systems,... Rather than bashing erlang like this, I'd have gotten a lot more out of an article that described how Haskell approached the problem,"
I'm not sure exactly where you saw the claim that Haskell is better at writing distributed concurrent systems. The Haskell distributed concurrency story is basically that there isn't one. There's a very early stab at adapting Erlang-style concurrency as a library, but there's nothing that I know of that is shipping and standardized that is a solution to that problem, and there's certainly nothing like Erlang, where a certain type of reliability informs the design of the language from top-to-bottom.
Someday I look forward to a Haskell that has stolen (ahem) Erlang's OTP but we're a ways away from that.
Also, clearly explaining Erlang's origins as focused on reliability, rather than "concurrency", is doing Erlang a favor, not criticizing it. I think more people would put up with its quirks if they realized that at least some of them come from this strong focus on a use case usually ignored by mainstream languages, rather than seeing it as an attempt at a mainstream language with a strong concurrency focus. (Some of its quirks are just bad, though.)
Jlouis (the author) is an advanced Erlang user who does know the language very well. He wrote etorrent, an Erlang bitorrent client in Erlang (which was also compared with combinatorrent, his Haskell version).
The article isn't about bashing Erlang, but changing the perspective under which it is observed. Namely, the language isn't about parallelism, but parallelism is a nice-to-have feature made possible by the concurrency constructs of the language, which themselves come from a need for fault-tolerance. I would certainly say he is right and I support this way of explaining things.
I'm a somewhat advanced Erlang user, and I enjoyed the article.
I am not sure where you got that competition with Haskell aspect, but I don't think it was ever the point.
14 comments
[ 2.9 ms ] story [ 32.3 ms ] threadThat was largely a semantic issue, which I responded with:
Parallelism: simultaneous calculations executing in the service of a single problem, usually with the goal of improved performance.
Concurrency: executions in the same time granularity, but not necessarily simultaneous. Also not necessarily in the service of the same problem, but some form of synchronization is required.
What I will add this time is that there are different kinds of parallelism, and I think he's considering "data parallelism" the real parallelism. I think this is a false distinction. What he calls "concurrency," I and others call either task parallelism (for distinct tasks that don't need to communicate much) or pipeline parallelism (for tasks that work on data in stages).
You can also classify parallelism. An example is to compare that you annotate parallel execution on the data structure (this is a parallel array) or on the control-flow (when operating on this array, do it in parallel). Another is if the parallel distinction is largely dynamic (worker pool) or static (we know we have 16 cores so the optimal communication structure assuming a double torus pairing configuration is ...). Yet another is to look at the data themselves. Are they a flat array or a nested tree structure (Nested Data Level parallelism is cool btw). And finally, to stop all the examples, we could look at declaratively making the code parallel and doing so explicitly yourself from primitives (think rolling your own database versus writing queries in SQL for an existing one - and hope the query optimizer can help).
The main point is that Erlang has taken a choice here. And that choice only covers some uses of what people may want to do. For the other things, people tend to use Erlang as a controller and then write their FPGA code and run it separately. This is what the original telephony switches were doing with Erlang.
Concurrency is a systems problem: design a system that safely runs n separate but interacting contexts of execution. Erlang's solution to concurrency--no communication between processes other than asymmetric message passing, no shared state other than the process registry--is a great solution to concurrency in terms of safety and distributon across multiple hosts, but it provides limited help to parallelism.
Erlang's reputation as a great language for concurrent programming on multicore processors is largely by accident--it has a great concurrency model for networks of distributed machines, and almost any concurrency model is better than mutexes and threads, but on multicore machines, Erlang wins only by default.
That's the same model of executions that MPI supports, and the most high-performance, parallel programs ever written have been written with MPI - scaling to thousands of processors across hundreds of nodes.
I feel that the people who are trying to define "parallelism" and "concurrency" are clashing with existing definitions from the already established high-performance computing community.
Parallelism - computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel").
Go to slide #13 in my presentation:
http://www.slideshare.net/nivertech/migrationtomulticore
Before introduction of SMP VM - Erlang was strictly concurrent, with only ability to write parallelism via distribution (i.e. multi-node systems). With SMP VM you can write single-node parallel code too. Not, that it's the best language for HPC, but you can always call C/C++, Fortran, CUDA/OpenCL from it using NIFs.
I'll say it does. beam.smp spins up multiple threads and I see them using more than one core regularly on my machines.
Rather than bashing erlang like this, I'd have gotten a lot more out of an article that described how Haskell approached the problem, possibly with contrasts to erlang.
But when you start off with at best nonsense "erlang's parallelism is not parallelism" you're going to lose people who do know something about erlang.
Here's the deal with erlang: The syntax throws some people off. If you will spend a couple hours learning the language (maybe a couple days) you'll start to appreciate the syntax and it will no longer be a problem.
Erlang is not as fast as bare metal languages like, say, C, but Erlang is fantastic for making programs in parallel and executing them on multiple cores or nodes or clusters of nodes with multiple cores.
I guess I should take the occurrence of these kinds of articles as a sign that Erlang is getting wider acceptance. That's great!
But please, cheer your language, don't bash the competition.
I'm not sure exactly where you saw the claim that Haskell is better at writing distributed concurrent systems. The Haskell distributed concurrency story is basically that there isn't one. There's a very early stab at adapting Erlang-style concurrency as a library, but there's nothing that I know of that is shipping and standardized that is a solution to that problem, and there's certainly nothing like Erlang, where a certain type of reliability informs the design of the language from top-to-bottom.
Someday I look forward to a Haskell that has stolen (ahem) Erlang's OTP but we're a ways away from that.
Also, clearly explaining Erlang's origins as focused on reliability, rather than "concurrency", is doing Erlang a favor, not criticizing it. I think more people would put up with its quirks if they realized that at least some of them come from this strong focus on a use case usually ignored by mainstream languages, rather than seeing it as an attempt at a mainstream language with a strong concurrency focus. (Some of its quirks are just bad, though.)
Jlouis (the author) is an advanced Erlang user who does know the language very well. He wrote etorrent, an Erlang bitorrent client in Erlang (which was also compared with combinatorrent, his Haskell version).
The article isn't about bashing Erlang, but changing the perspective under which it is observed. Namely, the language isn't about parallelism, but parallelism is a nice-to-have feature made possible by the concurrency constructs of the language, which themselves come from a need for fault-tolerance. I would certainly say he is right and I support this way of explaining things.
I'm a somewhat advanced Erlang user, and I enjoyed the article.
I am not sure where you got that competition with Haskell aspect, but I don't think it was ever the point.