> Lunar is my attempt to distill 40 years of programming language experience into the "best" language I can come up with
+1 for the name, but surely best language depends on use - graphics/os kernel/matrix programming, and environment - easily maintainable, extensible, learnable etc. There's a reason Python, Java, C are well embedded in certain areas.
It is a miserable disaster that machine learning and data science have largely settled on Python, because if you attempt to actually do anything in Python then it is far too slow. If you want to implement an alphazero-style training pipeline, you'll immediately discover that pytorch does not ship with an MCTS and a board game rules engine, and the only way to accomplish your task is to write these components yourself in C or C++.
More appropriate tools for the job exist. The state of the world we are in is largely dictated by what languages developers are forced to use because companies want hires to be highly fungible, because of library ecosystems, or because they want to ship code that runs on platforms that ban almost all languages.
Many or most machine learning and data science developers would probably find that Python + Numpy, Panda, SciPy, SciKit, TensorFlow, is far faster to develop in and probably faster and safer to compute than anything they could write themselves in C or C++.
Python is great for glueing stuff together - even when the libraries are in C/C++ (or Fortran for that matter).
In the realworld, the environment: availability of developers, library ecosystems, and the runtime platform requirements are nearly always much more important than the efficiency of the code.
> Many or most machine learning and data science developers would probably find that Python + Numpy, Panda, SciPy, SciKit, TensorFlow, is far faster to develop in and probably faster and safer to compute than anything they could write themselves in C or C++.
You are correct; they will find that they are completely unable to replicate papers from 2017 because almost all of the wall-clock time of their training pipeline will be spent running the Python interpreter.
CPython is what's slow, not Python itself. The main reason CPython is slow is that it's purely a bytecode interpreter and it makes no use of AoT or JiT compilation unlike the JVM, CLR, or V8 JavaScript engine. If anything, the rapid growth and adoption of Python gives me hope that we'll see something akin to the V8 engine for Python. Yes, things like Cython and PyPy exist but both have their issues.
Javascript had a reputation for being horribly slow before modern JIT compiled JS engines were created. I believe that around the time the V8 project began, a non-trivial number of browsers were using an AST-walk interpreter to execute JS (you can imagine how slow that was). If you look at some benchmarks for how V8 JS (for example Node.JS) performs relative to Java, C#, and C++, you will generally see that the performance gap between JS and Java/C# is comparable to the gap between Java/C# and C++. CPython, of course, is a lot slower.
There's nothing about Python that's fundamentally more dynamic than Javascript, though correct me if I'm wrong. Both languages have the ability to add arbitrary fields to objects at runtime (though I would say Python's syntax and object model tries to sway you away from doing it more so that JS), both languages have similar type systems and essentially fully first-class functions with closures, and both languages have an eval method that can execute any string as source code. The saying goes, "The squeaky wheel gets the grease", and for a long time Javascript was the squeaky wheel that everyone complained about but still had to use in order to target web browsers. That's what encouraged a huge company like google to invest resources into making a fast Javascript engine. I would not be surprised if something similar ends up happening with python.
I agree with your comment but don't expect improvements to Python runtimes because the high-performance libraries people depend on are tied up with the guts of cpython.
Edit: Actually, for alphazero specifically, it would not be enough to make cpython run as fast as V8. I have implemented the alphazero training pipeline in torch, which uses a dynamic language runtime with performance comparable to V8. After spending weeks optimizing my MCTS, rules engine, and game state encoding and decoding, I still found that the CPU time consumed by these things caused substantial underutilization of the GPU. But at V8 levels of performance this is more like "it takes 2x as long to train" rather than "it is not viable to train."
If I had done a better job of writing my above comment I would have made it clearer that this is not mostly about python-for-data-science. Most "right tools for the job" for other jobs are also points in tradeoff space that give up a lot of expressive power, ability to ensure correctness, performance, or all 3 in exchange for benefits that may only make sense over very short time horizons.
"Unfortunately I never finished this, and what code exists is completely broken and does not work at all. It might be better to start over with a fresh implementation."
Ah well. I guess we have nothing to experiment with
And it is the last line of last page of the document. Imagine reading through the whole thing and coming to the last page just to see the last sentence saying that.
I think it is more of a spec, and discussion of implementation ideas for a language than a manual of implemented language.
In defense, I would argue that many modern language design advancements DO relate to multithreading in some way. Rust’s borrow checker and linear/affine types can help with thread safety; OCaml is using algebraic effects to implement multi core behavior; GPU-related experiments (such as Formality [1]) are all about enabling more scalable and reliable parallelism.
That said, I don’t think that multithreading is the only interesting or worthwhile part of programming language theory these days. However, I think it is extremely fertile for innovation while also being somewhat isomorphic with other interesting correctness/performance/ergonomics problems.
18 comments
[ 3.7 ms ] story [ 23.2 ms ] threadhttp://www.archub.org/arcsug.txt
Right one : https://lisphub.jp/common-lisp/users/index.cgi?Essay%3AS-exp...
+1 for the name, but surely best language depends on use - graphics/os kernel/matrix programming, and environment - easily maintainable, extensible, learnable etc. There's a reason Python, Java, C are well embedded in certain areas.
It is a miserable disaster that machine learning and data science have largely settled on Python, because if you attempt to actually do anything in Python then it is far too slow. If you want to implement an alphazero-style training pipeline, you'll immediately discover that pytorch does not ship with an MCTS and a board game rules engine, and the only way to accomplish your task is to write these components yourself in C or C++.
More appropriate tools for the job exist. The state of the world we are in is largely dictated by what languages developers are forced to use because companies want hires to be highly fungible, because of library ecosystems, or because they want to ship code that runs on platforms that ban almost all languages.
Python is great for glueing stuff together - even when the libraries are in C/C++ (or Fortran for that matter).
In the realworld, the environment: availability of developers, library ecosystems, and the runtime platform requirements are nearly always much more important than the efficiency of the code.
You are correct; they will find that they are completely unable to replicate papers from 2017 because almost all of the wall-clock time of their training pipeline will be spent running the Python interpreter.
Javascript had a reputation for being horribly slow before modern JIT compiled JS engines were created. I believe that around the time the V8 project began, a non-trivial number of browsers were using an AST-walk interpreter to execute JS (you can imagine how slow that was). If you look at some benchmarks for how V8 JS (for example Node.JS) performs relative to Java, C#, and C++, you will generally see that the performance gap between JS and Java/C# is comparable to the gap between Java/C# and C++. CPython, of course, is a lot slower.
There's nothing about Python that's fundamentally more dynamic than Javascript, though correct me if I'm wrong. Both languages have the ability to add arbitrary fields to objects at runtime (though I would say Python's syntax and object model tries to sway you away from doing it more so that JS), both languages have similar type systems and essentially fully first-class functions with closures, and both languages have an eval method that can execute any string as source code. The saying goes, "The squeaky wheel gets the grease", and for a long time Javascript was the squeaky wheel that everyone complained about but still had to use in order to target web browsers. That's what encouraged a huge company like google to invest resources into making a fast Javascript engine. I would not be surprised if something similar ends up happening with python.
Edit: Actually, for alphazero specifically, it would not be enough to make cpython run as fast as V8. I have implemented the alphazero training pipeline in torch, which uses a dynamic language runtime with performance comparable to V8. After spending weeks optimizing my MCTS, rules engine, and game state encoding and decoding, I still found that the CPU time consumed by these things caused substantial underutilization of the GPU. But at V8 levels of performance this is more like "it takes 2x as long to train" rather than "it is not viable to train."
If I had done a better job of writing my above comment I would have made it clearer that this is not mostly about python-for-data-science. Most "right tools for the job" for other jobs are also points in tradeoff space that give up a lot of expressive power, ability to ensure correctness, performance, or all 3 in exchange for benefits that may only make sense over very short time horizons.
Ah well. I guess we have nothing to experiment with
I think it is more of a spec, and discussion of implementation ideas for a language than a manual of implemented language.
That said, I don’t think that multithreading is the only interesting or worthwhile part of programming language theory these days. However, I think it is extremely fertile for innovation while also being somewhat isomorphic with other interesting correctness/performance/ergonomics problems.
[1]: https://github.com/moonad/Formality
[1]: https://www.luna-lang.org/