His "true parallelism" means two (or more) actual simultaneous executions (threads, processes, etc.) occurring at the same time (redundant with simultaneous, for emphasis). This is contrasted with concurrency (see Pike's talk Concurrency is not Parallelism [0] for an example of how this line is drawn), which may or may not be implemented with "true parallelism". Coroutines, the example Gosling gave, often do not actually have simultaneous execution.
That Erlang forces some synchronization that other languages may not does not affect the ability to (with versions of Erlang since, what, 2007 or 2008?) make use of SMP for simultaneous execution of processes on separate cores or processors.
See also OpenMPI for an example of a distributed programming system that also can't share memory without copying it. Though in its case it was across physical nodes (and so the need to communicate was more obvious, they physically didn't share memory anyways). Still true parallelism.
And while MPI was designed for multi nodes, I’ve found that in many cases where a code base supports both MPI and openMP (shared memory with multiple threads in one process), the MPI version runs faster because each single threaded process knows that it owns it’s memory and there’s no need for synchronization.
Being explicit about the data being passed around while defaulting to “not shared” has benefits for making use of all your cores more efficiently.
Because anything described as "true X" is a fallacy and everyone has different views on what "true" is, what "parallelism" is and what "true parallelism" is. Sure, if you define "true parallelism" as "share memory between cores without copying it", then Erlang doesn't give you that.
It seems that James Goslings view of "true parallelism" is "let you take advantage of multiple processors", so with that frame of mind, Erlang does give you "true parallelism".
> Coroutines kind of magically sidestep some of the naughty issues in true parallelism. And for me, one of the problems with coroutines, which is why I haven't used them in a long time, is that they don't actually let you do or let you take advantage of multiple processors. You can't do true parallelism.
> But as soon as you've got one of these coroutine-based languages and you try to exploit multiple processors, if you're doing a lot of coroutine-type operations and you don't have enough processors, you're just saturating one processor.
I'm not sure there's too much debate on what parallelism is (simultaneous execution of 2 or more processes/threads). The broader debate seems to be on what concurrency is. Is it a design feature (in which case coroutines in a single threaded program offer concurrency) or is it an execution model (in which case it intersects with or is the same as the common definition of parallelism).
The other debate is on how parallelism (and concurrency) should be done. Shared-nothing or shared-memory? It'd be foolish to say that only one of them is parallelism, but it would be reasonable to debate the merits of each approach in different systems and circumstances.
I'm trying to name the version of parallelism that can share memory between threads, and so far I have tried "Joint Parallelism" and now lately "Atomic Parallelism" without much success.
I'm not sure it needs a special name. It's the default model that almost everyone learns first, shared-memory parallelism/concurrency. It's the whole reason that semaphores and mutexes were developed (to avoid contention for the same resource or resource set from multiple threads of execution).
Yes, but I also mean that parallelism to be atomic/non-locking. afaik only Java/C# can do this on a high-level (without going back to atomic arrays in C) but I still can't explain why, the best I found so far is:
"Many lock-free structures offer atomic-free read paths, notably concurrent containers in garbage collected languages, such as ConcurrentHashMap in Java. Languages without garbage collection have fewer straightforward options, mostly because safe memory reclamation is a hard problem..." - Travis Downs https://travisdowns.github.io/blog/2020/07/06/concurrency-co...
But it seems there is also some benefit to have a VM in addition to a GC for this joint/atomic parallelism.
Look up MISD. I think that’s the idea you’re getting at, there really isn’t a lot of hardware support for it which is why for more complicated structures you’re pushed towards higher level languages.
I wrote that. The problem I was getting at is that it's fairly easy to write various types of lock-free structures in native langues, but you almost always run into the "memory reclamation" problem.
E.g., you can look up something like a lock-free linked lists, the first of which were described decades ago. They have an algorithm to delete a node from the list, but what often isn't emphasized is that you can't just free that note by calling `free()` on it, because you don't know if any other readers might happen to have a reference to that node, and thus may access the underlying memory. If you free it, they might access freed memory.
In general, this is a hard problem to solve. You can wait, but how long is long enough?
Garbage collected languages solve this more or less automatically: there is no explicit free() but instead the GC determines when there are no more live objects and frees it for you.
This isn't unsolvable in native languages, but none of the solutions are as clean and tradeoff-free as you'd find in a GC'd language.
Historically, this is known as "multiprogramming". Multiple programs making forward progress inside the same computer, via interleaved or concurrent execution, the latter being called "multiprocessing".
Thanks, I agree with you overall. My comment was more focused on the (old and "well-used") adage of "X is not true Y" that gets used over and over again to look down on X, while the definition of anything "true" is most likely just personal opinion and is different depending on who you ask.
> The other debate is on how parallelism (and concurrency) should be done. Shared-nothing or shared-memory?
Obviously it should be "shared nothing" as it would allow you to horizontally scale your application and code will be much easier to develop and test. But in reality you inevitably need to have a shared state in one way or another. You could try to move that state into some dedicated service, like database, but it's not always performant enough (or even applicable). So you end up with shared memory because of performance.
>But if you call it "pattern matching," and it has less power than a regular expression, it feels misleading or like false advertising. But, as just a feature, I think it's great.
I always it was haskell that introduced the term...
The concept certainly predates Haskell by a few decades, but I haven’t found the origin of the phrase.
I tried to find a copy of this SNOBOL paper from 1973[0] to see whether it occurs there, but no luck so far. It doesn’t appear in the original COMIT paper[1].
Haskell copied a huge amount of its core concepts from ML, and even then, ML was very much a collection of prior experimental concepts with a new type system on top.
It’s a pedantic statement. Correct for the base language, but numpy supports 16, 32, 64, and maybe extended/128 bit too. It’s the kind of nonsense argument from language flamewars that has no relevancy for practical use cases.
Anyway, java does not have unsigned int, so there you go.
Edit: if he claims that int-is-float like in javascript, he’s just flat wrong.
It definitely isn't[1]. Python has both floats and integers. He's sort of right though that it doesn't have "true integers ... which ... improve performance" since Python's numeric types are boxed and not exactly fast (unless you use numpy).
Also, which "many other" languages have only doubles, aside from JavaScript?
Edit: I said Tcl, but that has the opposite problem: integers are fine, but floating point is problematic because everything is strings. Updated below to remove Tcl.
I don't think this is true of most languages; only for Lua that I know of. And it causes problem: if you send a 64-bit ID to Lua, which represents it as a double, it will subtly change the last few bits. Then when you go to look up something with that ID, you get the wrong item.
Tcl hasn't been string based for 20 years, misinformation is hard to eliminate. Since the release of version 8.0 in 2001, Tcl internally has value types for integer, bigints, double, boolean, lists, etc.. As long as you're careful not to cause the internal value to 'shimmer' into a string representation, Tcl maintains the value type as last used. Tcl happily converts values to strings as needed, giving the impression of 'everything is a string'.
I would guess he might be talking about how integers are objects in python, and stored in heap, rather than being stored as values directly on the stack. I haven’t looked, but it could also be in python that all numeric types are stored according to IEEE 754 before being unboxed to the indicated type.
can anyone expound on what he means when he says that lisp macros defined syntax but not semantics?
---
> The folks at Rust tried to do a decent job of fitting macros in the language.
> Other languages, like all of the Lisp family, managed to fit them in more gracefully, but they had a way of defining syntax where the syntax was almost entirely free of semantics. And in most languages, syntax and semantics kind of go hand in hand.
An if statement and a function call have different forms in C. They have the same form/syntax in lisp. It's like the power of everything is a file/bytestream in unix. Because everything has the same form (uniform interface) then if you write a tool it can be made to work on everything. Grep works with all processes because they all have a uniform output stream. Macros in lisp work with all constructs in the language. Usually the basic form has to be quite primitive to do this. If it is too primitive you end up with what is known as a "turing tar-pit". But people can fall in love with the power of this uniform interface so much that they do not realize that they have ended up with a tar-pit at a high level. This can happen with macros but it doesn't have to.
makes sense, it seems a fundamental tension within lisp itself as another commenter said "everything is eventually a list in lisp" then i guess macros have to be the same
maybe to support semantics "this is an if statement" could be some kind of metadata on the macro i suppose?
I am also not sure, but in compiler design semantic analysis includes checking to see variables match their types, and obviously lisp macros are partially so powerful because their is no type checking.
For me this was the biggest challenge when I was introduced to lisp: everything is just a list. Powerful, yes, but in other languages the ground felt firmer underneath my feet when I knew a class was a class, a struct was a struct, and array was an array, all with their own unique syntax. In lisp everything just looks the exact same. That is part of why it is so powerful, but at the cost of semantical references that can help users understand a lisp program.
I know, but syntactically speaking, when looking at a program, it is always just lists embedded in lists. That is the lack of semantic references I am thinking about. When I talk about being feeling grounded, I mean seeing a distinguishable syntactic structure that gives me a (semantic) reference point for what I am looking at.
I may be imposing my own views on his wording, but my interpretation is that his concern is related to the fact that macros have similar syntax to everything else in the language. Consider these two forms:
> (and a b c)
> (and* a b c)
Can you tell by looking that the the first is a macro and the second is a function? The first short circuits and evaluates only the argument forms it needs and the second always evaluates all three. That's a big difference, and not at all obvious from the syntax. In fact, it's hidden by the syntax, because the syntax is exactly the same for each.
Macros are part of the power of Lisp, but they change the normal rules of evaluation (the semantics) in ways that are not necessarily obvious unless you know the specific definitions behind each symbol in function position.
We can usefully be both a macro/operator and function under the same symbol:
This is the TXR Lisp interactive listener of TXR 268.
Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet.
1> (special-operator-p 'and)
t
2> (fboundp 'and)
t
3> (fun and)
#<intrinsic fun: 0 param + variadic>
4> (and 23 nil (prinl 42))
nil
5> (call 'and 23 nil (prinl 42))
42
nil
6> [and 23 nil (prinl 42)]
42
nil
7> [mapcar if '(nil t t nil) '(1 2 3 4) '(a b c d)]
(a 2 3 d)
8> (if t (prinl 'true) (prinl 'false))
true
true
9> [if t (prinl 'true) (prinl 'false)]
true
false
true
If we use the functional brackets, all arguments are evaluated the same way, and the leftmost value is treated as a callable object applied to the remaining values.
A symbol in the leftmost position of [...] is never treated as a macro (other than symbol macro) or special operator, even if it has a such a binding.
I find that one can invert pretty much every opinion that Gosling holds on a technical matter and be better off. It boggles the mind that people listen to him.
That's a nice interview. With the parts about Simula, I really feel like we should have some kind of history of computer science easily accessible, so that we don't do a complete circle every X years. It seems like every "new" feature has already been tested in an old language.
That’s not a helpful way of looking at it I think. Ideas don’t exist in isolation, and thus old experiments have to be carefully analyzed to extract what’s still true out of it. Often times it’s just cheaper to redo the experiment.
This is true across the board for engineering. It’s hard to build something that survives long prolonged contact with engineering realities. Computer architectures change. How things are done efficiently changes. The set of trade-offs we choose to make changes as the underlying business shifts or better business models are discovered. And heck, businesses can be cyclical for no better reason than humans are naturally cyclical in trends (see fashion)
It was very interesting interview. They write the source code, and a static type checker parses the program, constructs an abstract syntax tree, and checks everything it can. And then possible errors are highlighted right within a text editor.
"Modern developers use IDEs, like NetBeans, IntelliJ IDEA, or even Visual Studio Code. "
I'm curious how can Eclipse, one of the greatest java projects, be omitted? Then I saw the name of JetBrains at the end of the article, and realized that I just read another soft advertisement.
He says: "And it's really funny because JavaScript is essentially Java with the type declarations removed."
That's pretty glib and I think not very informative.
For more information you can read Sections 2 and 3 of this http://dx.doi.org/10.1145/3386327
Yeah, that seems like a strange sentence coming from a language designer who should know better. Just on the surface there is the differences of statically typed VS dynamic, class-based VS prototype-based, the implicit global state in JS, JS having first class functions and so on.
Longer quote seems weird as well
> To really get the performance improvements you see, it helps dramatically to have a statically-typed language. For dynamically-typed languages, like Python, it's really, really hard. And often, what people end up doing is adding annotations to the language so that you get languages like TypeScript, which is essentially JavaScript with type annotations. And it's really funny because JavaScript is essentially Java with the type declarations removed. So TypeScript is essentially Java with permuted syntax.
He seems to be saying that TypeScript leads to faster code execution as TypeScript has types, but fails to understand that TypeScript still compiles to JS and the browsers run JS without types, so should have the same performance as just writing JS directly (probably less performance, as TSC adds bunch of wrappers and other unnecessary stuff you would avoid if writing vanilla JS)
>He seems to be saying that TypeScript leads to faster code execution as TypeScript has types, but fails to understand that TypeScript still compiles to JS and the browsers run JS without types...
I'm not saying this is the case, but isn't there a greater opportunity to optimize TypeScript code over JavaScript due to the presence of types? Isn't this the same thing as a statically typed program being compiled into assembly (untyped) ?
There is a greater opportunity, but it hasn't happened in practice. And even with Typescript's types, I doubt that it would make much of a difference because the type system is deliberately not sound and the types can't really be relied on. Typescript is a great way of reasoning about a program, but it's not meant as a runtime execution type system.
Yes. Another common example is that in languages without pointers, the lack of aliasing enables optimizations that are impossible for languages with pointers.
Also, C compilers generate efficient code by assuming undefined behavior cannot happen. (This sometimes corrupts buggy programs, because the type system can't guarantee undefined behavior isn't triggered.)
I was hoping to hear more about his opinion on Kotlin, since it's, basically, replaced Java on Android, but there's little there outside of saying it runs on the JVM.
Has it really? (Totally honest question, I don't follow that space well.) If Some Random Company is starting a new Android app today, is it more likely to be in Kotlin than Java? That's cool, if so.
Not really. It just another language, but, IMHO, threads have gotten A LOT better through Kotlin’s Coroutines. That said, the libraries have gotten a lot better and the new UI toolkit, Compose, addresses a lot of shortcomings, but it’s still early.
Kotlin has a lot of features aimed at mitigating the pain of working with Java APIs, such as extension functions. So you can sort of fix the API in a library or in your own code. (But it doesn't help with the lifecycle or reducing the number of layers.)
I'm not so sure about that, yes it's an important language and has oficial support from Google, but by the amount of usage we can say Dart ( in Flutter ) also "replaced" Java on Android.
Don't get me wrong I HATE Java on Android and everything GUI related, but I'm don't think Java will be replaced anytime soon.
Not sure I agree with that. More on "Android’s Kotlin-first approach" is at https://developer.android.com/kotlin/first. I haven't heard the same about Flutter and Android's new UI toolkit, Compose, is written in Kotlin.
74 comments
[ 3.5 ms ] story [ 163 ms ] threadErlang: "Hold my beer"
That Erlang forces some synchronization that other languages may not does not affect the ability to (with versions of Erlang since, what, 2007 or 2008?) make use of SMP for simultaneous execution of processes on separate cores or processors.
See also OpenMPI for an example of a distributed programming system that also can't share memory without copying it. Though in its case it was across physical nodes (and so the need to communicate was more obvious, they physically didn't share memory anyways). Still true parallelism.
[0] https://blog.golang.org/waza-talk
Being explicit about the data being passed around while defaulting to “not shared” has benefits for making use of all your cores more efficiently.
It seems that James Goslings view of "true parallelism" is "let you take advantage of multiple processors", so with that frame of mind, Erlang does give you "true parallelism".
> Coroutines kind of magically sidestep some of the naughty issues in true parallelism. And for me, one of the problems with coroutines, which is why I haven't used them in a long time, is that they don't actually let you do or let you take advantage of multiple processors. You can't do true parallelism.
> But as soon as you've got one of these coroutine-based languages and you try to exploit multiple processors, if you're doing a lot of coroutine-type operations and you don't have enough processors, you're just saturating one processor.
The other debate is on how parallelism (and concurrency) should be done. Shared-nothing or shared-memory? It'd be foolish to say that only one of them is parallelism, but it would be reasonable to debate the merits of each approach in different systems and circumstances.
What would you suggest?
"Many lock-free structures offer atomic-free read paths, notably concurrent containers in garbage collected languages, such as ConcurrentHashMap in Java. Languages without garbage collection have fewer straightforward options, mostly because safe memory reclamation is a hard problem..." - Travis Downs https://travisdowns.github.io/blog/2020/07/06/concurrency-co...
But it seems there is also some benefit to have a VM in addition to a GC for this joint/atomic parallelism.
E.g., you can look up something like a lock-free linked lists, the first of which were described decades ago. They have an algorithm to delete a node from the list, but what often isn't emphasized is that you can't just free that note by calling `free()` on it, because you don't know if any other readers might happen to have a reference to that node, and thus may access the underlying memory. If you free it, they might access freed memory.
In general, this is a hard problem to solve. You can wait, but how long is long enough?
Garbage collected languages solve this more or less automatically: there is no explicit free() but instead the GC determines when there are no more live objects and frees it for you.
This isn't unsolvable in native languages, but none of the solutions are as clean and tradeoff-free as you'd find in a GC'd language.
Obviously it should be "shared nothing" as it would allow you to horizontally scale your application and code will be much easier to develop and test. But in reality you inevitably need to have a shared state in one way or another. You could try to move that state into some dedicated service, like database, but it's not always performant enough (or even applicable). So you end up with shared memory because of performance.
I always it was haskell that introduced the term...
I tried to find a copy of this SNOBOL paper from 1973[0] to see whether it occurs there, but no luck so far. It doesn’t appear in the original COMIT paper[1].
[0]: https://dl.acm.org/doi/10.1145/361952.361960
[1]: http://www.catb.org/~esr/comit/comit-paper.html
https://en.wikipedia.org/wiki/Gosling_Emacs
I don't think that's correct for Python!
Anyway, java does not have unsigned int, so there you go.
Edit: if he claims that int-is-float like in javascript, he’s just flat wrong.
No, it’s not. In the base language Python has an int type that is separate from its float type.
See numbers.Integral and numbers.Real
Also, which "many other" languages have only doubles, aside from JavaScript?
[1] https://docs.python.org/3/reference/datamodel.html#the-stand...
I don't think this is true of most languages; only for Lua that I know of. And it causes problem: if you send a 64-bit ID to Lua, which represents it as a double, it will subtly change the last few bits. Then when you go to look up something with that ID, you get the wrong item.
Most languages don't have that problem.
"datePublished": "2021-08-06T15:45:40+03:00", "dateModified": "2021-08-06T15:45:40+03:00",
---
maybe to support semantics "this is an if statement" could be some kind of metadata on the macro i suppose?
For me this was the biggest challenge when I was introduced to lisp: everything is just a list. Powerful, yes, but in other languages the ground felt firmer underneath my feet when I knew a class was a class, a struct was a struct, and array was an array, all with their own unique syntax. In lisp everything just looks the exact same. That is part of why it is so powerful, but at the cost of semantical references that can help users understand a lisp program.
> (and a b c)
> (and* a b c)
Can you tell by looking that the the first is a macro and the second is a function? The first short circuits and evaluates only the argument forms it needs and the second always evaluates all three. That's a big difference, and not at all obvious from the syntax. In fact, it's hidden by the syntax, because the syntax is exactly the same for each.
Macros are part of the power of Lisp, but they change the normal rules of evaluation (the semantics) in ways that are not necessarily obvious unless you know the specific definitions behind each symbol in function position.
A symbol in the leftmost position of [...] is never treated as a macro (other than symbol macro) or special operator, even if it has a such a binding.
The fact that he inflicted Java upon the world is proof enough.
This is true across the board for engineering. It’s hard to build something that survives long prolonged contact with engineering realities. Computer architectures change. How things are done efficiently changes. The set of trade-offs we choose to make changes as the underlying business shifts or better business models are discovered. And heck, businesses can be cyclical for no better reason than humans are naturally cyclical in trends (see fashion)
I'm curious how can Eclipse, one of the greatest java projects, be omitted? Then I saw the name of JetBrains at the end of the article, and realized that I just read another soft advertisement.
The last one: https://news.ycombinator.com/item?id=27886575
Longer quote seems weird as well
> To really get the performance improvements you see, it helps dramatically to have a statically-typed language. For dynamically-typed languages, like Python, it's really, really hard. And often, what people end up doing is adding annotations to the language so that you get languages like TypeScript, which is essentially JavaScript with type annotations. And it's really funny because JavaScript is essentially Java with the type declarations removed. So TypeScript is essentially Java with permuted syntax.
He seems to be saying that TypeScript leads to faster code execution as TypeScript has types, but fails to understand that TypeScript still compiles to JS and the browsers run JS without types, so should have the same performance as just writing JS directly (probably less performance, as TSC adds bunch of wrappers and other unnecessary stuff you would avoid if writing vanilla JS)
I'm not saying this is the case, but isn't there a greater opportunity to optimize TypeScript code over JavaScript due to the presence of types? Isn't this the same thing as a statically typed program being compiled into assembly (untyped) ?
Also, C compilers generate efficient code by assuming undefined behavior cannot happen. (This sometimes corrupts buggy programs, because the type system can't guarantee undefined behavior isn't triggered.)
my main pain point with it was more the API rather than Java (e.g. the activity lifecycle, the 20 layers of compatibility stuff, contexts everywhere)
did this clear some of that up?
Don't get me wrong I HATE Java on Android and everything GUI related, but I'm don't think Java will be replaced anytime soon.