> Terra is a low-level system programming language that is embedded in and meta-programmed by the Lua programming language.. Like C/C++, Terra is a statically-typed, compiled language with manual memory management. But unlike C/C++, it is designed from the beginning to be meta-programmed from Lua.. In Terra, we just gave in to the trend of making the meta-language of C/C++ more powerful and replaced it with a real programming language, Lua.. Terra programs use the same LLVM backend that Apple uses for its C compilers. This means that Terra code performs similarly to equivalent C code.
Hanrahan and his students developed Brook, a language for GPUs that eventually led to NVIDIA’s CUDA. The prevalence and variety of shading languages ultimately required the GPU hardware designers to develop more flexible architectures. These architectures, in turn, allowed the GPUs to be used in a variety of computing contexts, including running algorithms for high performance computing applications, and training machine learning algorithms on massive datasets for artificial intelligence applications.
Although it's true that replacing LLVM like-for-like would indeed be expensive, I feel like you missed GP's point. To make the easier to debug, what you want is to replace LLVM with something simple, like a plain interpreter. And writing one (or adapting one) would be nothing like this cost.
Rust is experimenting with a codegen tool called cranelift which powers wasmtime. The plan is to use it for debug builds. There is also a backend implementation in the works using gcc jit and a dotnet backend as well
Zig is actually trying to get rid of LLVM for default builds, though they intend to keep it as one of the "backends" for the forseeable future.
The problem is that LLVM is very slow (as it applies lots of optimisations) and heavy, and that makes a compiler slow even in debug mode.
D has 3 backends, LLVM is used by the LDC compiler. Then there's DMD which is the official compiler (and frontend for all three), and GDC which uses GCC backend. D code typically compiles quite a bit faster on DMD, but the code is more highly optimised with LLVM or GCC, of course. It's a great tradeoff specially since all compilers use the DMD frontend so it's almost guaranteeed that there's no differences in behaviour between them.
Both Terra and Extempore [1] implement the same basic idea, which is provide a way to generate performant machine code on the fly but keeping a language surface that feels like you are working with a dynamic language (Lua or Scheme correspondingly). I think Scopes [2] could be mentioned here too. The concept is really cool and different from JIT compiling in that the developer is given the tools to generate machine code, as opposed to some "magic" JIT compiling happening opaquely behind the scenes.
That being said, I don't think either of these languages is practical for real world projects for several reasons: lack of (or low quality) documentation, bus factor (lack of developers), tiny communities, lack of tools, lack of libraries, etc etc etc.
> I don't think either of these languages is practical for real world projects
Maybe not yet, but prior work by this research team eventually lead to CUDA. Terra may be useful as a productivity DSL for some high-performance computations, e.g. listed in their papers:
our DSL for stencil computations runs 2.3x faster than hand-written C
our serialization library is 11 times faster than Kryo
our dynamic assembler is 3–20 times faster than Google’s Chrome assembler
> Terra is a workhorse and it gets the job done.. Having first-class code generation capabilities has been really nice, especially with built-in support for things like parsing C header files, vectorization, and CUDA code generation. The language is stable and generally doesn't have many surprises. Unlike, say, C++ or Rust, there aren't as many corners to hide odd behaviors.
"I am here to warn you, traveler, that Terra sits on a throne of lies. I was foolish. I was taken in by their audacious claims and fake jewels. It is only when I finally sat down to dine with them that I realized I was surrounded by nothing but cheap plastic and slightly burnt toast."
do you understand that "use one language to emit another language that is then immediately compiled" is literally like hundreds of languages now? numba/cupy/pytorch/jax/julia/etc/etc/etc. they all implement both a high level IR that can be transformed via the high level language (python) and a JIT/compiler/whatever.
so like why would i ever use this thing terra+lua+"5 guys maintaining it in a basement somewhere" thing?
Achieving high performance and power efficiency on future architectures will require programming systems capable of reasoning about the structure of program data to facilitate efficient placement and movement of data.
Are the Stanford researchers in a basement? The lab's previous work lead to CUDA. Does that earn them any consideration? How about lanl.gov using the language?
I don't know how to explain this to you because you and everyone else around here worships at the alter of academia (especially HYPSM academia) but the answer is absolutely 100% yes. There is literally nothing coming out of any edu lab, even stanford, that has any relevance in this same industry (ML/AI compilers). To wit: there is no company running absolutely anything on top of legion.
> The lab's previous work lead to CUDA
This is like saying a sketch of a car led to the car. No. CUDA is the result of thousands of working engineers years labor, not academia, not academic aspirational hacking.
> How about lanl.gov using the language?
"using the language". using how? every single group at lanl is using the language? they've migrated all of their fortran to using the language? they're starting new graduate programs based on the language? you really have no clue how low the barrier to entry here is - any undergrad can start an association with lanl (or any other national lab) and just start writing code in whatever language they want and suddenly "lanl is using the language". you don't believe? me go look for any number of reports/papers on julia/go/python/javascript/etc/etc/etc coming out of lanl and ornl and jpl and etc. how do i know this? i'm personally on a paper written at fermilab based primarily around task scheduling in JS lolol.
> there is no company running absolutely anything on top of legion
Does Nvidia count?
Supercomputing is on the verge of a breakthrough—computing at the exascale, where a billion billion calculations per second will be the norm. That’s 10 times faster than current supercomputing speeds... Los Alamos scientists, along with colleagues at five other institutions (the Nvidia corporation, the University of California–Davis, Stanford University, SLAC National Accelerator Laboratory, and Sandia National Laboratories) created a parallel programming system to do just that. The system, called Legion, sifts through an application to determine which tasks can run in parallel, or simultaneously, to save time and boost computing efficiency. [1]
> This is like saying a sketch of a car led to the car.
The ACM Turing Award (computer science equivalent of Nobel Prize) committee believes otherwise.
Beginning in the 1990s, he and his students extended the RenderMan shading language to work in real time on the newly available technology of graphical processing units (GPUs). The GPU programming languages that Hanrahan and his students developed led to the development of widely used standards, including the OpenGL shading language that revolutionized the production of video games. Subsequently, Hanrahan and his students developed Brook, a language for GPUs that eventually led to NVIDIA’s CUDA. The prevalence and variety of shading languages ultimately required the GPU hardware designers to develop more flexible architectures. These architectures, in turn, allowed the GPUs to be used in a variety of computing contexts, including running algorithms for high performance computing applications, and training machine learning algorithms on massive datasets for artificial intelligence applications. [2]
From SIGGRAPH 2004 slides, Stanford's research on Brook was sponsored by Nvidia, ATI, IBM, Sony, U.S. DARPA and DOE. [3]
> > there is no company running absolutely anything on top of legion
> Does Nvidia count?
I don't understand? To prove that NVIDIA ships legion you sent me a link to a LANL post? How does that make sense? Show me a product page on NVIDIA's domain to prove to me that NVIDIA uses this in a product.
> The ACM Turing Award (computer science equivalent of Nobel Prize) committee believes otherwise
You seem to not get what I'm saying: my firm is position is academia doesn't understand absolutely anything in this area. Zero. Zilch. Nada. And absolutely no one in the industry cares either. So given that position, why is this relevant?
The only thing academia is good for is a talent pool of hard-working, smart people. We take those people and then completely retrain them to do actually useful work instead of research nonsense. The vast majority of PhDs coming from academia to industry (yes even from Stanford) literally are horrible software/hardware engineers. Many of them stay that way. The good ones (at least in so far as they care about having a successful career) learn quickly. That's how you get CUDA, which is a product worth a trillion dollars.
Look I've already told you: you workshop at an alter and you've also clearly never worked at a Stanford or an NVIDIA or a LANL. You'll never be convinced because... well I have no idea why people need mythologies to worship.
> Legion is research (where _future_ products originate)
my drawings of spaceships are also where future spaceships come from. it's plausible right? therefore my drawings of spaceships are valuable right?
> Would you care to share the names of some software/hardware engineers worthy of emulation by academia
sorry that was a mistype - i meant to say no one in the industry cares either. engineers in the industry do understand things because they're working on the things every day.
did you miss the part where i've written/collaborated-on research papers? on grants that are $10MM+. it's meaningless because the goals of research are not "make something useful".
I guess one of the issues is that because Lua is tailored for embedded use case, it lacks appeal as a general purpose software development ecosystem and thus its usage is marginal.
It’s been adopted in game engines to handle story logic, so the people writing the game can be decoupled from the people writing the engine, and don’t have to code in a low level language against a constantly moving target.
And I would argue that a lot of the staying power of World of Warcraft came from the Lua based addons democratizing the UI. Blizzard co-opted a lot of features from best of breed addons back into the base game. (That and negative pressure in the economy to keep from having every new character wielding the third best sword in the game due to deflation, but that’s a story for a different thread)
Lua is nice but it being embedded in many different places means that there will often be environment specific context that's injected for you to interact with, but any editor tooling won't know what's injected
Not necessarily true.
LuaLS, the most popular LSP, uses LuaCATS (Lua comments and type system) annotations.
You can place annotation files (type stubs) in a directory and the LSP will pick them up.
I use this for a project of mine where I embed a Lua interpreter where I made several additional Lua modules available.
I ran into this recently with OBS scripting. I don't really like Lua, so I wanted to try making a Typescript-to-Lua toolchain specifically for OBS scripting. Well OBS injects a ton of stuff into the Lua environment, and I'd have to find a way to mock all of that in Typescript. Sadly, I don't have time for it, and I've chosen a different product than OBS because I don't really want to invest so much time learning a new language to accomplish something with free software when I can purchase something that just works for $40. My time is way more valuable, and this is the only time I've ever needed to code something in Lua. Yeah, OBS supports Python too, but I dislike Python even more than Lua.
I just needed a program to capture a portion of the screen under my mouse and stream it as a webcam device to Zoom, so I can screen cast a portion of my gigantic workstation desktop (6480x3840) to people with 1920x1080 screens, while following my mouse cursor around.
OBS was one stop along my journey to get there, as there's some Lua and Python scripts that achieve something similar, but not quite exactly what I wanted with multi-screens, so I began trying to modify the script to do what I needed it to do.
In the end I'm going with something called "SparkoCam" which achieves this easily with no hassle, but it costs $40. Since $40 is less than what an hour of my time is worth, it was kind of a no-brainer to abandon the hassles of learning Lua (a many hour detour) and fiddling with OBS Studio to get what I wanted. I have no real use for Lua outside of this, so it would be time wasted on this one project.
I felt the same way, hesitating to use a hardware RT music platform that is scripted in lua because "omg another lang", but I couldn't find an alternative. I bit the bullet and ... man it really is easy to use, forgiving and absurdly productive. You should reconsider, because learning lua is so easy, with so many resources out there for any question ... and then you know a "real" language and not just some OBS tech.
I simply don't have any reason to use Lua now. I replaced OBS with software that just works, and that saved me countless hours of hacking a new language to do something that may not even work how I need it to. $40 is money well spent, and Lua will have to wait for another project, if one ever comes up. And if OBS hadn't injected so much extra stuff I'd probably settled on a Typescript-to-Lua toolchain to do the work, because I just don't really have time for context switching to languages I'll only ever use for a single project.
It depends. In Debian's [0][1] benchmarks, JS is clearly ahead in a handful of the benchmarks, but in the rest, Lua is either on-par or faster. They're within a shout of each other.
There are some places where Lua has surprisingly poor performance. However, in almost everything else, it's one of the fastest around. Luajit is still one of the fastest JITs in existence.
In the real world, rather than constraints of a benchmark, Lua tends to be on-par from what I've seen. When you're not cold-cacheing everything, its memory use is lower. But it's standard library is ISO-C, and that's about it. No batteries. And the others beat it soundly there, whilst being close to the same performance. Which will mean they will just about always win out.
Thanks. I calculated the geomean of the speed-up factors based on the average results (cpu secs) per benchmark. Overall, Lua 5.4.6 is factor 5.1 slower than V8 v23. Only in k-nucleotide Lua is slightly faster than V8, but in n-body Lua is even factor 26 slower.
It's also a completely different approach. Probably additional performance could be gained by implementing JIT support for additional LuaJIT bytecodes. But we cannot predict how significant this gain would be. If the concept has limitations, more money likely doesn't change much. We can also compare with the old Mono 3 version developed by a rather small company (Xamarin, before they were bought by MS), which follows yet another approach than V8 or LuaJIT, and which achieves about the same performance as V8; see https://github.com/rochus-keller/Oberon/blob/master/testcase....
Unfortunately, the comparison pages don't actually show every test, or even the same run, as the main pages. You'll find a lot of differences between what I linked, and what is linked here.
Overall, the JS/V8 version was twice as fast as the Lua/LuaJIT version. LuaJIT was only faster in the Mandelbrot benchmark. The recent PUC Lua version 5.4.1 is even ten times slower than the V8 version at the time. More recent V8 will likely be even faster. The results also demonstrate that one can get a pretty false impression when just looking at single microbenchmarks.
I think it's a stretch to call it underrated. Isn't it the defacto language for embedded scripting stuff? I've seen it used as a mini language in production in game engines, databases, and trading systems.
On the other hand, if the article is saying that it deserves to be more highly rated in the non-embedded space, I think I would need more than a handful of syntactical niceties to explain why it deserves to be in the same conversation as Python, Ruby, etc...
I was infected by the Lua mind virus 15 years ago and I have not recovered.
I simply don't see any reason not to use it .. for everything.
I've built games with it. I've built GUI's with it. I've built scientific analysis tools with it. I've used it as a total solution for previously-impossible embedded problems. I've used it for desktop applications and mobile apps. I've automated entire production lines with it. It has huge appeal as a general purpose software development ecosystem - but those who know how to do this, tend to keep it to themselves for some reason.
It can do everything.
There's just nothing I want to do with computers that I can't, legitimately do, with Lua - or its VM. Or, LuaJIT. Or, all of the above and a bit of luarocks built in. Throw some luastatic in while we're at it, bundle it all up as a .deb, and nobody needs to know a thing.
Its just so versatile. I hope someone builds an OS around it some day, making everything and anything the user wants to do, accessible through a Lua module, or some smart libffi wrappers around a handful of .so's, somewhere ..
In my retirement, I'll take antirez' LOAD81, turn it into a proper Lua editor, add integration with luarocks and luastatic and maybe a shit-tonne of IPFS, and then there won't be any need, ever again, to deal with anything else.
This happened to me about 6 months ago. (I didn't seek it out, just ended up working on something where lua was involved). So thanks for the warning. I didn't realise it could last for 15 years or more. The big thing for me is how easy it is to make something that feels like a DSL while it's still just Lua when you debug it.
I've found different ways .. if its a simple Lua situation, where you just want to debug pure Lua code, Zerobraine Studio is a pretty good environment to do it and comes with a debugger.
I've tried various other Lua debuggers over the years, these are useful:
Otherwise, I just use gdb if needed (JIT/FFI stuff) and printf's.
However, I don't really need to debug much these days - more just testing. Its also kind of important to have automated tests and run them frequently with every major change.
Another thing I use frequently in my Lua projects, which I think cuts down a lot of the common bugs that might require debugging, is the lua_enumerable.lua utility module:
It looks like Lua's conditionals, for loops, and function definitions terminate with the "end" keyword. Is this considered more readable or does it bring some other benefits besides helping the interpreter parse code?
Many people love Lua, so I suspect there is a good reason for this.
Algol (and Pascal) originally did the same thing that C did, except that you used `begin` and `end` instead of `{` and `}` when you needed a compound statement as a branch etc.
Modula changed it so that all structured constructs implicitly introduced a compound statement, and terminated one where appropriate (e.g. on `else`), but this still required some end marker for the last one.
If the question is why `begin` and `end` and not `{` and `}` - for one thing, Algol was designed to be the language you'd write reference implementations of algorithms in for publication, so readability was more important than terseness. For another, it predates ASCII, and not all charsets in use at the time even had `{` and `}` as symbols.
It depends on which alternative you're thinking of.
- }, like C: Lua uses this to terminate tables. You could use the same token for both purposes (Perl5 does) but the reduced redundancy comes at some cost to both readability and syntax error reporting, which are especially important in languages aimed at driveby programmers, like Lua.
- ): Lisps generally just use ) to terminate every construct, reducing redundancy further.
- different end tokens for each construct, like Ada and the Bourne shell (endif/end if/fi): improves readability and error reporting further, but requires more reserved words and is wordier.
- end, like Lua: intermediate among the above choices.
- indentation, like Python: seems to work well enough in Python, and especially helps beginners who are therefore not permitted to indent their code incorrectly, and adds a maximum amount of redundancy for opening and closing constructs (every line is explicitly tagged with its syntactic depth) without reducing readability. However, this aspect of Python was novel when Python was introduced, remains controversial, and has not been widely adopted in new languages.
A funny thing about Lua's syntax is that it's not newline-sensitive, but semicolons are optional. This is valid Lua:
function x(a, b)f(a)f(b)g = a b = a end
Wherever two things are juxtaposed that can't be part of a single statement, it tries to parse them as two statements. This syntactic looseness means that some syntax errors that would be easily detected in semicolon languages aren't easily detected in Lua. This increases the penalty for removing additional syntactic redundancy.
I've been more and more intrigued by fennel lately, and I've been meaning to dive into the functional end of the pool. Any recommendations for how to get started with it, especially if coming from a python background?
Also interested in resources here -- familiar with python, go, rust, but never used a lisp.
As a learning experiment, I was able to use the fennel website to convert my Hammerspoon config from lua to fennel, but I still struggle to add new functionality. Certainly have not delved into macros, but this is where much of the power lies AIUI.
I have learned it (somewhat) by playing around in TIC-80 and Löve2D (including LoveDOS).
Both Lua and Fennel are tiny languages. You can skim all the documentation in a short evening. Fennel does not even have its own standard library, so keep the Lua documentation close at hand.
I appreciate the simplicity of both Lua and Fennel. There are some ugly parts, but all languages have those.
Janet (later project by Fennel's creator) is a nicer language that is also portable and embeddable, and has a nice standard library, but Fennel runs anywhere Lua runs. Janet is great just on its own for scripting and not sure I would want to use Fennel for that anywhere Janet is available. But as a way to script games in e.g. Löve2D it seems like an excellent choice.
class MyTable is export {
has @.data;
multi method new(@data) {
$.new: :@data;
}
method render {
table :border<1>,
tbody do for @!data -> @row {
tr do for @row -> $cell {
td $cell } };
}
}
[in defence of raku, while it truly is marmite wrt Ruby / Python, there is a very loyal core of smart people that love the freedom and expressiveness it offers]
> there is a very [tiny, very] loyal core of smart people
FTFY ;) Honestly, I love Raku - it's almost like Racket of the non-sexp world - but the community is microscopic, and the development is slow. It's a shame because, solely based on features and in terms of design, it's a serious contender for the most powerful dynamic language out there. It's packed with interesting features (junctions, phasors, traits, quote/unquote macros (soon), typed variables and subsets, coercible types, augment, OO system with multiple inheritance, interfaces, generics, multi-subs and multi-methods everywhere, custom operators including circumfix, regexes and grammars, etc, etc...) out of the box and it's actually really impressive how all these features work together in a cohesive design.
Unfortunately, the underlying implementation seems a bit chaotic; some features are either not fully implemented or specified, the performance could be better, and so on. So, while I love and still use Raku for almost all my scripting, if I were to start a serious project, I think I'd go with Common Lisp instead. CL has fewer features out of the box, but many of the missing ones are one `ql:quickload` away (or you can use CIEL), while the underlying implementation (SBCL in my case) is diamond-solid.
---
Aside: that `render` method could be sweetened a lot! For example:
thanks for the code examples - personally I prefer mine (but that's my opinion)
one liners have their place - but I like to avoid compression for the sake of it
to put some numbers on it there are 146 folks listed on the main IRC chat (plus a bunch more (79) via the Discord channels) right now and 1.5k members of the sub-reddit
Sure! I just thought if we're talking about "Syntactic Syrup of Ipecac" :D we can as well take it to the extreme, just to show how sugary Raku can really be (if you want).
As for the community size, I'm on Discord for a bunch of languages:
- Haxe: ~800 (users on the server)
- Pharo: ~300
- Rust: ~10k
- Scala: ~900 (might not be the gathering place of the community, would need to check)
- Nim: ~700
- GToolkit: ~160
Taking your numbers, the community is about as big as GToolkit, which is a niche distribution of a niche language (Pharo Smalltalk). I think that's enough to keep the lights on and even to occasionally produce something really impressive. At the same time, it's not enough to consistently polish all aspects of the language and the ecosystem: with a community of that size, you will always be running into "not implemented yet"/"no bugfix for years" problems. Which is fine in some settings, but you need to keep this in mind.
It's the other way around for me, and I don't understand why people like Python's system so much.
Moving code around and changing scope in other languages is just a matter of adjusting the braces, but with Python I need carefully reindent all affected lines. It's error prone and annoying.
People say (and it is true) that you read more than write, so your careful indenting is for that case. I don't find python easier to read than c; I find the verboseness quite annoying actually. Plus what you said in usability for the coder. It is nice as teaching language but beyond that I have no clue why people use it; if you read the modern AI code in it, it's not very readable at all as that is not about python but rather numpy and torch etc which could've been done in another language which might have been nicer for this purpose. Now (check for instance gpt2 or so) it's verbose typing/reading for stuff that's not readable for someone who doesn't understand the theory anyway.
It’s hard to tell since there’s no changelog, but it looks like there are at least some improvements ongoing other than just simple bug fixes. Some of the changes in August for instance look like more than just for correctness.
I have really enjoyed working with Lua when it's come up and I think it's an extremely good language. In particular, as Noë says, its interface for embedding into C/C++ is clean and flexible. There are performance concerns, but considering how widely Lua is used for game logic in high performance video games clearly those concerns can be addressed.
That said, I think there are other warts that can be confusing and make the language difficult. The Nil-Terminated Arrays section points out one oddity about `nil` - but there are many others. There are a number of confusing corner cases (or were - I haven't used it in a while). As an example, if you have a list of values and you pass that list into a new table, the resulting table may be different than if you add each value in that list in order.
I also think that the flexibility of embedding means that you are often learning the Lua <=> C interface as much as you are learning Lua. So jumping from deploy to deploy is more confusing than most languages.
Nevertheless I fully agree the language is underrated!
The Lua C API is fantastic from a design perspective, but as you have discovered, it is rather error prone. Fortunately, there are many wrapper libraries that take care of all the stack manipulation. For example, I'm using https://github.com/ThePhD/sol2 to interface with C++ and the API is incredibly ergonomic.
It might be fantastic in other ways; there are multiple ways an API can be good. For example, it could be very expressive and flexible, or it could be especially amenable to formal proofs with things like TLA+ even if it's error-prone in human hands, or it could facilitate very high performance. Even on the error-proneness axis, an error-prone API might still be less error-prone than the alternatives.
It is a low-level API for creating bindings to other languages and as such typically isn't used directly. (Well, unless you are writing C :)
What I find fantastic about the stack-based API is that it exposes very few interpreter internals. As a result the authors may drastically change the implementation without breaking the public API. Most importantly, they hide the internal object representation and as a user you only ever deal with plain C types, like integers, floats, C-strings and user data pointers. As a consequence, you don't have to think about memory management because all types are passed by value! In fact, someone could theoretically write a Lua implementation with reference counting and the API wouldn't have to change. It really is a very smart and elegant design.
You can only make a C API for interacting with a precise garbage collector that ergonomic. And, to be honest, I wouldn’t call the situation described by the GGP particularly problematic—you messed up memory accounting and you got a memory leak. In most garbage-collected runtimes, if the runtime messes up memory accounting, at best you get immediate memory corruption, at worst you get memory corruption much later in some unrelated part of your heap. So this is pretty mild all things considered. Of course, if you’re working in a language with greater metaprogramming capabilities than C, absolutely do abstract the stack into something more manageable.
How is it fantastic, it’s just barebones stack interface thrown in your general direction.
Any integration code that doesn’t use an ad-hoc helper library looks like assembly listing (not even in disguise, cause it basically is bytecode but in C).
The problem is, if we are talking about Lua API, typically you are on the other side of “embedded in another project”, as an embedder. My reply shortly above goes into some details.
(from elsewhere) It is a low-level API for creating bindings to other languages and as such typically isn't used directly. (Well, unless you are writing C :)
Yeah, unless. But even under this assumption, wrapping handles to values would be much more easier in a language-specific wrapper, because it wouldn’t have to watch out for stack overflows etc and solve the exploding stack problem in general in naive api-user-side loops. This thread started as Lua stack API is better than Python’s, iirc. But Python API already looks like a high-level wrapper doing exactly that, so… what was the point again?
I’m a little concerned that everyone talks high matters here without providing examples or at least looking into “final particles”. Cause that’s where claims get tested against reality. I spent too many years in deep Lua and its mailing list to miss something obvious that everyone knows.
Returning handles would introduce subtle memory management issues because the garbage collector wouldn't know which objects it should keep alive. With the stack-based API the user can only ever interact with objects that are on the stack and thus are kept alive by the garbage collector. The Lua authors explain this in the following article:
https://queue.acm.org/detail.cfm?id=1983083
I would agree that the Lua C API itself is not very eegonomic, but the wrappers/bindings it enables absolutely are! For example, integrating Lua in C++ with sol2 is an absolute joy.
Yes it could pin accessed objects in some arena, like idk... "stack", for the duration of the call. And release it after an "invocation" returns, or if the host requests it explicitly. It's almost like my example already mentioned it near "everything but ref’d things disappears here" (/s).
I understand how gcs work and their nuances. My only problem was with the "fantastic" part. Lua stack is not "not ergonomic", but absolutely shit to work with, and "there's wrappers" is not an argument, cause they are not a part of distribution but external projects ought to fix the mess.
> it’s just barebones stack interface thrown in your general direction.
I do not do a ton of low level development, but my impression is that you do not want a demanding interface. You want an interface that is straightforward, doesn't have complex pre & post conditions, and doesn't crash. Leaking 4 bytes per frame is bad but crashing would be worse! You may think you want a library that refuses to function at all unless its called perfectly, but once you're in the trenches you may change your mind.
To my mind, it's a very good sign when there are popular "adapter" libraries. It means the underlying code (Lua) can be used many ways and people have found best practices for different approaches. When I used it I was not working on a game and it would have been worse for us if Lua's interface was "built for games." Being very bare bones and low level allows flexibility and that flexibility ensures a healthy ecosystem.
#define T lua_gettop(L)
…
int t = T;
…
assert(T == t + expected_inc);
…
lua_pushvalue(L, -2); // note: this is b
lua_remove(L, -3); // old b
// eventual realization that
// -index is fragile
// and rewrite to
int start = T;
int a = ++start;
int b = ++start;
luaL_checkstack(L, 10, "…");
…
lua_pushvalue(L, b);
lua_remove(L, b);
lua_call(L, …);
luaL_ref(L, …);
…
lua_settop(L, t - 1);
If you have ever done this, idk how that feels flexibility. I’d rather get handles to a and b and directly call a function with them as like:
And avoid all this stack-in-your-mind and stack-in-your-int nonsense, which is completely unnecessary in C and I fail to see where exactly it is flexible, compared to the short snippet above. Stack is just how vm sees it, it is not any special “for you” thing, it’s literally implementation shining through.
I’d like to see a specific example (low-key, I left Lua) on how its stack provides more flexibility or anything, because in my humble experience it does not, absolutely, do anything like it. It just makes you play these off-by-one games all day for no good reason.
Like I said I am no low-level C guy, but my impression is that the existing structure is allowing you to break up the setup however you like across functions / calls / etc (just keep the context) instead of keeping many different contexts. The approach you roughed out is fine as long as you can 100% swap any arg for any other when you make the call - but I think often that's not true? Or if it is you need to pass all the info. So I suspect you are drilling down on a simple case that elides the problems that will arise in the complex case.
You can, ofc, write your Lua wrapper like your example - all in one place. It reduces the risk of misuse. But you could also do it differently - which is the flexibility that I think is a strength.
> but considering how widely Lua is used for game logic in high performance video games clearly those concerns can be addressed.
It’s ok, but should be much better.
Once upon a time Supreme Commander used Lua. Perf on consoles for SupCom2 was catastrophically bad. We switched to Kore which was effectively Lua but way faster (10x?).
Kore was bought by Havok and rebranded into Havok Script. Havok got bought by Microsoft. I’m honestly not even sure if you can still buy it today. It’s not listed on their website.
LuaJIT is pretty good, but some platforms might not allow it. iOS technically didn't allow it when I used it before, but Apple either didn't _actually_ care or just looked the other way.
My humble opinion is that most applications that need scripting should just embed JavaScript.
It has a massive advantage in the existing ecosystem. People don’t always love it, but millions of them already know it. You don’t need to re-learn array indexing and nil termination quirks and the dozens of other things where Lua is subtly different in ways that you’ll find out at runtime.
There are JS engines in all sizes, from tiny interpreters to world-leading JIT performance monsters. And most operating systems nowadays ship with a high-quality engine and embedding framework that you can access without installing anything, like JavaScriptCore on the Apple platforms.
Do your users a favor and just use the boring standard language instead of making them learn a completely new one that doesn’t offer meaningful practical benefits for them. It’s probably not as much fun for you, but you’ll save time for both the users and yourself in the end.
On one hand, you do have JavaScript ubiquity, on the other hand, you have the JavaScript "WTF" equality and typing conversion matrix, "WTF" scoping, and dozens of other foot guns that Lua doesn't have. Some people would rather everyone learned Lua subtle differences and wrote less buggy code. I've had to use for the web for 20 years, and if I have any choice, I would avoid it.
Anyway, every language has some fans, and I don't mean to dismiss them, but ubiquity is not the top priority, all the time.
There is the equality gotchas on JavaScript but then you have 1-indexed arrays in Lua, which is a footgun that I know exists, I try to avoid shooting myself in the foot with it but I keep doing it.
It is beyond a footgun, more like a footgun concealed as a shoehorn.
Either way I find embedding a Lua VM into a project so much easier than embedding a Javascript one.
Indexing, I give you this one... I think it was intended as some kind of a gift to a "non programmer", a choice many other older language made, by the history decided to go another way, unfortunately for Lua. Still, I haven't had any issues adjusting to this one, personally. It seems kind of like Python whitespacing, I think I forget about it.
The only people saying "WTF" about Javascript are people who haven't taken the time to learn how to use it effectively. There are "WTF" things about every language, not just Javascript.
Noobs are going to find the footguns in any language. Every language can be misused. I've been using Javascript over 20 years and have never thought it was any worse than any other language. Yes, it's dynamic, and yes, you can do stupid things with that dynamism, but for someone who actually knows how to use it, it's quite easy to work with.
I recently started using Lua and I'm not impressed at all. It's just another language, it's really up to the programmer what can be done with it, just like Javascript.
This is the common arguement used by people defending languages with tons of footguns.
If this is true, why not write everything in C++?
Javscript includes so many footguns that people know are bad (see the billion of examples of different types being added and getting random answers). Every langauge has some difficult parts, pretending that means any amount is fine is crazy. Javascript has earned its reputation.
>(see the billion of examples of different types being added and getting random answers)
Almost every "footgun" people cite about Javascript is something that would never be written as working code, by any competent programmer - they are "WTF" in that the person writing the "gotcha" has to go way outside of normal programming behavior to get the "WTF" result. Most of these "WTF" examples should be taken with a grain of salt.
If you're expecting to "WTF" about Javascript's type inference/coercion, that works according to some logical rules.
This matrix should be pretty easy to understand. Yes some things might seem weird like [[]] == false, and [1] == true, but nobody should be writing code like that, it's just illogical to do anything this way and I don't consider it to be a "gotcha". "Just because you can, doesn't mean you should" can be said about footguns in every language.
Yes, there are "WTF"s, but just like if I were to write a C++ "WTF", it wouldn't be something that most people would be tripped up by unless they were specifically looking for a way to say "WTF" about the language.
These "javascript sucks" internet arguments have persisted for decades, and it's always complaints about the language that are citing unreasonable examples to prove a point that a programming language has some rough edges. Every programming language has some rough edges.
Javascript maybe has more footguns than some other languages because it is so dynamic, and the dynamism has its benefits too that other languages don't have. I do like it and I use it effectively, YMMV. People just seem to be specifically mad at Javascript because it's so popular and was made so by being the defacto standard for web browsers. It's part jealousy and part frustration that their favorite programming language isn't the one that web browsers include by default.
Agreed. I think a lot of the "classic" footguns are picked up by current-day linters/AI/code-assists anyway, so even if you do somehow end up writing [[]]==false legitimately (explicitly or implicitly), you'll probably get a squiggly warning about it on your IDE immediately warning you.
For better or worse, JavaScript is the Lingua Franca of modern programming - you have a high-performance runtime environment pre-installed on basically anything, and readily installed on greybeard boxen if you are that way inclined . It is my "go to" for all hobby projects, frontend and backend. So versatile and a genuine joy to code in.
This sounds like you haven't dealt with much moderately complex code in JavaScript. It sounds like you've never run into a situation where innocuous-looking code like
if (isAuthenticated)
{
...
}
behaves strangely and it takes you hours to track down in debugging, only to find that somewhere else in the code, isAuthenticated was assigned an array and it was assumed it could be treated as a Boolean that would be false if and only if the array was empty. Turns out sometimes the value is [[]] and it gets treated as false unexpectedly.
In any reasonable language, even a dynamically typed one, this should just be a type error because it's not a Boolean, but JavaScript decides to do something almost certainly unwanted. This is the real source of the complaints, not the fact that you can write “[[]]==false” on the console and laugh at the results.
>only to find that somewhere else in the code, isAuthenticated was assigned an array
That isn't "moderately complex code", that's completely stupid code and you deserve the grief if you are willy-nilly setting a boolean as an array. Maybe learning the difference between a bool and an array is something you should have done earlier in your programming journey.
Also love how you assume I've never worked on "moderately complex code" as a slight on me, when I've been writing software for over 40 years - just because I don't do stupid shit like assigning an array to a boolean variable does not i dicate I've "never worked on moderately complex code". That's some kind of gatekeeping troll.
>any reasonable language
I'm really not sure how stupid you have to be to assign an array to isAuthenticated. No, it is not something I have ever done in any language, because I'm not an idiot. YMMV.
> This is the real source of the complaints,
The complaints come from people doing stupid shit to themselves. It was easily avoidable and they got mad at the language?? Should we blame the language for every stupid thing people do with it? Will you also blame English for letting you write an obviously stupid comment?
Lua feels and acts like a worse javascript. Yes the runtime can make the memory footprint really small, but manipulation and validation of the crappy list/table structures makes the implementations miserably large. I've only written about a million lines of it (literally), so I'm pretty set in my opinion.
Okey, so the first thing everyone should learn is not to use "==" but use "===". Or, perhaps, just use TypeScript, right? I think that's what people call "effective JavaScript" these days.
Quite a lot can be built reliably with Javascript following only "use ===". YMMV. Typescript doesn't solve much unless you use it religiously (and few people really do), it still allows for all kinds of "gotchas" if you want to write "gotchas" with it.
Js needs no forwards which in Lua turn into ugly “local foo; … … … foo = function()…end”. All functions are “hoisted” by default.
Js also uses “export”s or “module.exports[…]” instead of returning a symbol table, which allows cross-referencing.
Js has a function scope (var) together with local scoping.
Js doesn’t allow accidental shadowing in the same scope. Lua: local x; local x; silently creates two variables, good luck debugging that closure.
Js type coercion is only insane in the parts where operation itself makes zero sense. Please don’t tell me that ![0] or [] + {} makes sense to you or that you use them in other languages. Or that s+n and n+s seems safe everywhere except in js.
Js has a ternary operator and ?? ?. ??= operator group. Lua only has that “and-or” crutch and “not X” which returns true only for nil and false.
Js “…arr” is a first class citizen while in Lua “…” decays into its last value if you want to follow it.
Js (normally) ordered properties system is much more convenient in dev and debug time than an unordered mess of meta-hashtables.
For cases where you can realistically include a full v8 in your project, yea - that seems fairly reasonable nowadays.
Javascript took an extremely high-complexity, high-cost, and nearly-all-effort-in-just-two-impls-for-over-a-decade route to get to the current (stunning) performance though. How do small-scale embedded interpreters stack up? That's what many things that aren't Electron apps will be using.
I ask this honestly, to be clear - I'm not sure how they stack up. I know Lua has historically targeted and dominated this area so I'd expect it to win, but I have not seen any data and I largely agree - JS is a massively better development environment now, and I would love to choose it. Everything I can find pits it against v8, and that's kinda like comparing the performance of a bike against a train - they are so different in many situations that it's meaningless. Otherwise I'd probably choose WASM today, or maybe docker containers where performance and platform needs isn't an issue.
QuickJS has somewhat higher requirements, but also a great candidate for embeddable mobile and desktop applications, and is well-regarded. https://bellard.org/quickjs/
This probably makes more sense in 2024 but for many years JavaScript was not a serious option. I wasn't until the mid-late 2000's that it started to get traction as an application language and had any way to run it outside of a web browser.
Lua as an embedded language already had over a decade head start.
Netscape had javascript running server-side in 1995, only a couple years after Lua was created. So no, Lua did not have "over a decade head start", it had 1 year.
Microsoft also had server-side JS, and still do. Active Server Pages (ASP) were scriptable with javascript since 1996. Microsoft's JScript.net has been around since 2002 (even if not well supported or maintained by M$, it still works). I've written web back-ends, Windows desktop applications, and even .dlls with JScript.net as far back as 2003.
Javascript has been Adobe's scripting engine for Photoshop since 2005. Sony chose Javascript for Vegas Video's scripting language in 2003.
Javascript has so many uses outside of a web browser, and has for a very long time.
Yes but for the first 10 years (1995 - ~2005) JavaScript was not taken seriously as an application language in most dev shops. Active Server Pages were overwhelmingly coded in VBScript. JavaScript was used in little snippets in "onBlur" attributes to do client-side validation of fields on a form, pop up "alert" boxes, and enable submit buttons. It wasn't until XMLHttp and prototype.js and jquery and stuff like that came along that mainstream developers started to understand what JavaScript was capable of.
I guess I wasn't a "mainstream" developer in the 1990's? Before XMLHTTPRequest, I was using iframes to asynchronously load content into the browser, essentially accomplishing the same thing before "AJAX" was a thing. We were doing long-polling before there was websockets. I was using Javascript and CSS to create "DHTML" SPAs before XMLHTTPRequest, with Javascript for front-end and back-end. You either know the potential of the tools that are available, or you don't.
But I guess my anecdotal experience doesn't matter, because I wasn't a "mainstream" developer? From my point of view, everyone else was missing out on using one language for front-end and back-end.
Context switching has a real cost in terms of developer burnout, and it's something I've happily avoided for 25 years.
"most people should use the things I use and am familiar with" is a common opinion but you should realize that between Javascript and Lua, Lua is the boring standard language in this context.
Also there are whole books written on the quirks you need to master with Javascript. Lua has its rough edges but it's a much simpler language, so there are far fewer of them.
Doesn't matter if it's widely known, it honestly is much nicer to work with than Javascript and is being used as it is intended, instead of being a half-baked language designed in a hurry, to work in a browser, that got repurposed as a general purpose scripting language later.
Lua has a ton of sharp edges, they just don't manifest themselves as much as with JavaScript since JS has a different "typical use case" and aren't as often discussed. Writing user facing UIs, with all that it entails, is a great way to uncover and experience all possible sharp edges.
And a lot of the issues with JavaScript just wouldn't matter for an "embedded scripting engine" use case imo.
Lua 5.2 is 15000 lines of portable ANSI C. On amd64 the Debian default configuration of the interpreter compiles to a 950KiB executable that strips to 224K. I think reduced configurations can go below 100K. It doesn't start extra threads, demand a thread of its own, or have tricky memory management demands. What JS interpreter would you recommend for cases where that's a good fit? Something that's less than a megabyte and can be compiled to run on weird architectures.
This does look great! Unfortunately it looks like it'll be hard for me to build it; it's yet another project that got fucked over by the Python community's irresponsibility.
I was looking at the example build configuration scripts on the project home page. Not somewhere linked from the home page, actually inline code on the page. Today! Literally as I'm writing this comment, it says:
Lua used to be very popular for embedding because the runtime was minuscule so it could run in very constrained environments (like old consoles). This is mostly a non-issue these days though.
JS and Lua are very similar feature-wise and I imagine Lua hasn't gotten the massive JIT runtime optimizations that JS has, so JS seems a far better choice these days.
Once you reach LuaJIT levels of performance though it's not like you need more. If you're doing a huge amount of heavy lifting in your embedded runtime then you've probably designed the application wrong.
The fat better arguments are that it's more common and there are more libraries.
Yeah that is a good point, I also imagine lua is far easier (build/linking) to add into your application compared to pulling V8 or JavascriptCore.
So if you only need a small language to do a couple thousand lines of code in than it still might be a better choice. But for larger projects JS still seems better due to the ecosystem and knowledge available.
But if you compare some games like baldurs gate 2 where most of the event logic and quest state is in the scripting language, then maybe it is worth spending the time to add JS. But if it is like Civilization 6 where only small stuff like UI and I/O mapping is in the scripting language then it is a different matter.
The problem I have with Javascript is that, for some reason (mostly, my lack of practice), looks really difficult to me.
I'm no developer, I do Linux and some bash and Python, and Lua looks "familiar" to me, so I can quickly pull up something simple.
With Javascript everything looks so alien to me, and it's a pity because there is lots of things that run Javascript (functions in the cloud and the like).
I disagree for the same reason -- how quickly the language can be picked up and used productively.
Due to Lua's small footprint, a novice user can skim through the Lua docs and walk away with a useful mental model for most of the language.
This is not the case for Javascript, which has far more syntax and many duplicated subtly-different features (var/let, class/plain objects, this definition depending on lambda syntax). Not something easily grasped by a new user.
And sure, there are many Javascript snippets floating around and its possible to pick up some subset of Javascript by osmosis. However, this approach leads to a "poke it with a stick until it works, while understanding nothing and slowly going mad" UX from what I've seen. Which is not something I would aim for in the kind of productivity tools embedded scripting languages are used in.-- I find these tools are far more satisfying and productive to use when you have a simple mental model which clicks in place after a bit of effort up front, letting you work after without having to consult docs for every step.
(Of course this is not an issue if you're targetting only developers proficient with Javascript as users, but this is not the case for most applications outside of the webdev space.)
I've had fun embedding Lua a couple times, but as with wasm I almost never have a case where I need to run new code without being able to recompile the whole binary, so I don't get to use it often.
The most fun was when I made a game engine ("engine") in C++ that used Lua for all the game logic. The fast debug cycles are very useful there. Maybe I should still do that for Rust gamedev, since Rust notoriously compiles slowly.
The second-most fun was an IRC bot that could reload its Lua logic without disconnecting. Nowadays I'd write a bouncer and just let the main process restart. (Or use wasm, a wasm IRC bot would be good...)
The third-most fun was using Lua as a Turing-complete configuration language. If I needed an array of something, I just made Lua make an array, instead of generating Lua code.
One of the least fun was the Blender plugin that exported 3D models as generated Lua. If I did that one again today, I'd just use MsgPack or some ad-hoc binary format.
I can't say I appreciate Lua any more after reading this. It only reinforced my annoyances with it.
It's been a while since I wrote any Lua but I didn't think it was intuitive at all. Why are arrays tables, why are they nil terminated and why do they start at 1.
I don't think it's underrated either. It's very popular for writing scripts in (older) game engines.
I might be able to provide at least some insight into that.
One of the things I found while implementing my language was that the ordering of elements in a hash table is essentially random. Literally random if you mix in some randomness into the hash function's initial state in order to harden it against collision attacks.
Many languages guarantee insertion ordering. It's a useful property to have, makes testing easy and deterministic and it's intuitive for users. A simple way to implement that is to put a perfectly normal array of values into the hash table, and then have the buckets which the keys hash into contain indexes into that array. That way the indexes of values remain stable no matter what the keys hash to.
So a sufficiently advanced hash table will have an array of values inside it. I suppose some language designers took one look at that situation and decided to just merge both types already by using hash tables for everything. For example, PHP also did that. Associative arrays, they call it.
CPython's hash table (dict) evolved to have an array of values indexed by a hash table of indices for performance reasons, but later decided to guarantee the insertion ordering that this permitted for iteration.
"Associative array" is just the term used for hash tables (dicts) in Awk, bash, Tcl, and, most relevantly for PHP, Perl4. It doesn't specifically mean that the language lacks a separate container type for small-integer-indexed arrays. It just means that that particular container type can be indexed by something other than small integers, usually strings.
No, that's an implementation detail which isn't actually true of the associative arrays in all of these languages. "Associative array" is the name for the interface, not the implementation. In theory it could be a hash trie or something, although I'm not aware of any implementations of any of these languages that uses anything but a hash table (which is, yes, fundamentally dependent on using an array).
> In addition to space savings, the new memory layout makes iteration faster.
> Currently, keys(), values, anditems() loop over the sparse table, skipping-over free slots in the hash table.
> Now, keys/values/items can loop directly over the dense table, using fewer memory accesses.
The "dense table" is the array of values. The "sparse table" is the actual hash table, only the values index into the values array.
I thought the insertion ordering was implemented at around the same time as a result of it. The data structure ensures it pretty much for free, seemed like the obvious next step. It appears I was mistaken about that.
They did implement insertion ordering at that time, but they didn't want to commit to adding it to the official language semantics until having enough experience to know whether they wanted to back out the change to the implementation which provided that guarantee for free.
You described how every language without hashtable-as-an-object works. The only difference is when you address fields.
But I fail to see the logic behind seeing this and merging array/table ideas. Feels like one of these irrational decisions you make in a dream.
And Lua doesn’t use hash table to store arrays. Its tables have two different parts (hash, indexed array) with foggy distribution logic which reflects into “# vs ipairs” issue. This merge is purely synthetic and doesn’t project any underlying implementation idea.
Merging the concepts of arrays and maps was very popular in the 1980s and 1990s. I can count at least Awk, Tcl, Lua, JavaScript and PHP as languages that went with this approach.
Looking from today's perspective, I also fail to see the logic behind it: separating arrays and dictionaries does not make your implementation more complicated than having a single backing data structure for both and taking care of all the special cases that arise from such a monstrosity.
The best explanation I can think of is that this was just a fad nobody questioned too much. Back in the 1980s and 1990s, there was a strong distinction between "scripting languages" and general-purpose dynamic languages (like Lisp and Smalltalk). Scripting languages were supposed to be used to write small scripts to automate stuff. Type-safety was not something that any mainstream language considered back then (it was a thing in niche languages like Ada, SML and Eiffel), and speed was not a concern for scripting languages.
A common philosophy (though I never seen it clearly stated) for scripting languages seems to have been to "be liberal and try to work with whatever the programmer does".
So you shouldn't require to declare variables in advance (that's a lot of hassle!) or throw an error if a function is called with the wrong number of arguments.
And what if the programmer tries to add an integer and a number written as string? Just coerce them. Or better yet, make everything a string like Tcl does. Or at the very least, don't be a spoilsport and have separate types for integers and floating point numbers!
Ok, so what if the programmer tries to use a string index? We cannot coerce non-numeric strings! Well, why don't we just unify arrays and dictionaries like we've just unified all our number types? Now you don't have to worry about invalid indexes since all indexes are converted to strings! And if the programmers want to use 0-based, 1-based, 2-based or 42-based indexes they can do whatever they want! They can even have non-contiguous arrays or mix arrays with associative dictionaries. Total freedom.
This thought process is completely imaginary. I suspect not a lot of thought was given to this type of flexibility at all back in the day. It was just a widely accepted assumption that scripting languages should be flexible, free and relaxed. And merging arrays and dictionaries (or "all arrays are associative arrays" as it was often called back then) was just one more idea that naturally fit into the zeitgeist, like type coercion and "assignment to an unknown variable sets a global variable by default".
Not to my understanding. What is called "array" in the language is indeed associative, but lists (heterogeneous vectors, like Python's) have always been here and distinct.
But otherwise, insightful post. Awk is also the first thing that came to mind, with its funky arrays and "undefined variables are initialized on first use to the empty string or 0 depending on the context".
This fad was also helped, I think, by the memory architectures of the day. Without modern multilevel caches and ridiculously expensive cache misses, it might have seemed reasonable to converge on a one-size-fits-all structure. The cost of jumping around a relatively compact structure like a hash table probably seemed immaterial at the time.
For the same reason why (standard) arrays are associative in PHP and JS, as well - because, from a certain perspective, a regular array is just an associated array with consecutive integers as keys, so if you're trying to keep things simple, it seems like a straightforward unification.
> why are they nil terminated
It follows from them being associated arrays, and Lua's pervasive use of "nil" meaning "this thing is missing".
I’m not parent, but for me the question is not literally why did they choose 1-based indexing. That’s easy to answer. The question is really: why haven’t they realized by now that the disadvantages of 1-based indexing far outweigh the benefits when it comes to programming, and changed it? Backward compatibility is a valid reason, but lots of other people have tried 1-based arrays and reverted to 0-based once it becomes clear that 1-based arrays causes more problems than it solves. Even humans who count starting from one will soon want to use 0-based the moment they need to add two array indexes together; we don’t want to have to start from 2 in that case, right? Early versions of the famous Numerical Recipes in C started with 1-based arrays, but removed them, and got slightly better. :P
Did you mean, "Julia never found any widespread adoption and remains an obscure, niche language"?
Fortran was the world's top programming language until the new millennium and is still among the top ten most popular. Arrays in Fortran can start from any index; the default choice is 1, which can be overridden with literally one keystroke: 0.
It is a misconception that a JS array is "just an associated [sic] array with consecutive integers as keys". Even the early (most readable) editions of the ECMA spec have a whole set of carve-outs that belies that claim.
> It's very popular for writing scripts in (older) game engines.
Exactly, it seems popular in places where the alternative is even less usable (e.g. write your own C++ plugin or thing). For that it's fine.
For example, there is a lua plugin for x-plane and it's great. I've done some minor things with it even. But that didn't make me a fan of the language. It's fine for simple things. But a bit limited.
The x-plane API is pretty low level and written in C++. So something that sits on top and is a bit more accessible is great. There are a lot of lua scripts available that you can run in there that are pretty nice. But the more complicated ones get pretty hairy quickly. Makes you long for something more structured.
I don't have the time, but I did consider working on adding a websocket API at some point to make it easier to integrate with from e.g. a web page or some kind of standalone process. I think they are now planning to add that feature themselves so there's no need to work on this anymore. And they have a low level UDP API currently that exposes all sorts of things. But a websocket API with some good client libraries for different languages would make a lot of sense.
It's my language of choice for letting users customize an app, but the last client wanted Python because many people are already familiar with it and 'Scriptable with Python' looks better from the marketing point of view(
I love Lua as a language; even if it can be a bit cumbersome at times, its simplicity and low barrier of entry is refreshing. I'm currently building a full-stack web application with https://redbean.dev in Lua and it's mostly been a joy.
Luau [1] developed by Roblox is a backwards-compatible fork of Lua 5.1 with some very significant improvements to speed and QoL [2]. It also mostly alleviates the nil-terminated array problem.
Pretty sure in a year or two the ratio will flip to where people will pick up on articles that are not assisted by LLM. Does this post read like it's been written completely by a human? And the models will probably get better until it's mostly indistinguishable.
LLMs and bots will probably be the thing that leads to a second version of the internet being created. Eventually the thing we currently know as "the internet" will be nothing but bots and LLMs continually regurgitating in a giant 100-billion-watt-consuming feedback loop.
An LLM would have written a much more convincing argument that "Lua is underrated".
This blog has just three posts, none with significant content. I was curious how this article even wound up on HN, until I realized that it was submitted by the blog's author: https://news.ycombinator.com/user?id=nflatrea
But seriously, they do have some interesting quirks which make sense if you think of it (and maybe Lua as a semi-contemporary got its ideas from there), specifically, hashes are pairs of keys (even/odd) in a flat array... and of course objects are just key/values, but with some values being names and functions...
There was a time when this approach was common for dynamic languages in general. PHP and, to some extent, JS also come to mind.
I think it's clear in retrospect that it makes more sense to keep lists and maps separate even in a high-level dynamic language that doesn't care much about perf, and any purported gains in simplicity from having a single data structure are illusory because of footguns. But perhaps it wasn't so clear before we had enough experience with this kind of design.
Then again, Python had a clean separation between lists and dicts from the get go, and it predates Lua as well as PHP and JS...
These days, yes, but this particular design point predates both LuaJIT and Lua popularity as a gaming scripting language specifically. I remember first seeing it sometime circa 2002 - I believe that was Lua 4.0? - and I was stricken by how much it reminded me of PHP and JS in this regard.
But yeah, my point is that one of the most common arguments against using associative arrays as the data structure is that perf overhead is unacceptably high if you use it as a list, but we've learned how to implement it in a way that can adapt to this use case at runtime just fine a loooong time ago. But regardless of perf, it just makes more sense to keep sequences and maps separate, not because of any specific technical reason, but because it's easier for humans to reason about things and model them that way.
Meh. I don't get the love Lua gets on HN.
At least version 5.1, I didn't try the others.
No first class support for OOP, you got to use tables instead.
No "continue" keyword to go to the next iteration of a loop. So you have to use some ugly elseif.
No try catch!
It's a dynamic language so everything goes. A function will return nil or a number or whatever you wish.
A function can also returns any number of variables, so obviously some moron will abuse it: looking at you Blizzard intern (you must have been an intern right?) that thought returning 20 variables on the combat event log was a good idea. Ever heard of tables???
The LSP can't do miracles when there are almost no rules so auto-completion is more miss than hit, which is quite shocking in 2024.
The IA is helpful sometimes but create subtle bugs some other times. A simple uncaught typo will create a hard to find bug (yesterday copilot auto-completed myObject.Id instead of myObject.id, there went 20 minutes of my life trying to find why some code that was running solidly for weeks was now failing silently).
So all in all, Lua is fine to write small imperative scripts of a few hundred loc.
Anything bigger and more complex you should run away IMHO.
I never realized C# was such a well designed language until I tried Lua.
For some people not being locked into one form of OOP or any OOP at all is a huge advantage.
> No "continue" keyword to go to the next iteration of a loop. So you have to use some ugly elseif.
Yeah, that sucks. Thankfully Luau fixes that and basically all annoyances I have with Lua. https://luau.org/syntax
> I never realized C# was such a well designed language until I tried Lua.
Lua and C# have fundamentally different design goals. C# sucks for embedding, you have to include a huge runtime. Godot still hasn't figured out how to support C# on web builds. Lua values minimalism and flexibility. That obviously means it lacks some quality of life features of bigger general purpose languages.
Mandatory plug for Redbean[0], a standalone Lua webserver (a single-file that runs natively on six OSes!), and Fullmoon[1], a web framework built on top of it.
Not exactly mainstream, but so simple and elegant.
368 comments
[ 3.0 ms ] story [ 481 ms ] thread> Terra is a low-level system programming language that is embedded in and meta-programmed by the Lua programming language.. Like C/C++, Terra is a statically-typed, compiled language with manual memory management. But unlike C/C++, it is designed from the beginning to be meta-programmed from Lua.. In Terra, we just gave in to the trend of making the meta-language of C/C++ more powerful and replaced it with a real programming language, Lua.. Terra programs use the same LLVM backend that Apple uses for its C compilers. This means that Terra code performs similarly to equivalent C code.
From Pat Hanrahan's research group at Stanford, https://amturing.acm.org/award_winners/hanrahan_4652251.cfm
Papers: https://terralang.org/publications.htmlCan it use anything else (as an option), e.g. Lua? That would be useful during development/debugging thanks to faster iteration and memory safety.
> 1.2k developers have produced a 6.9M line code base with an estimated price tag of $530M.
The problem is that LLVM is very slow (as it applies lots of optimisations) and heavy, and that makes a compiler slow even in debug mode.
D has 3 backends, LLVM is used by the LDC compiler. Then there's DMD which is the official compiler (and frontend for all three), and GDC which uses GCC backend. D code typically compiles quite a bit faster on DMD, but the code is more highly optimised with LLVM or GCC, of course. It's a great tradeoff specially since all compilers use the DMD frontend so it's almost guaranteeed that there's no differences in behaviour between them.
That being said, I don't think either of these languages is practical for real world projects for several reasons: lack of (or low quality) documentation, bus factor (lack of developers), tiny communities, lack of tools, lack of libraries, etc etc etc.
--
1: https://extemporelang.github.io/
2: https://sr.ht/~duangle/scopes/
Maybe not yet, but prior work by this research team eventually lead to CUDA. Terra may be useful as a productivity DSL for some high-performance computations, e.g. listed in their papers:
2021 HN thread, https://news.ycombinator.com/item?id=27334065> Terra is a workhorse and it gets the job done.. Having first-class code generation capabilities has been really nice, especially with built-in support for things like parsing C header files, vectorization, and CUDA code generation. The language is stable and generally doesn't have many surprises. Unlike, say, C++ or Rust, there aren't as many corners to hide odd behaviors.
Source: https://erikmcclure.com/blog/a-rant-on-terra/
> Please note: The entire "windows" portion of that article has largely been resolved at this point, because I fixed it myself.
> Terra programs use the same LLVM backend that Apple uses for its C compilers
lol is this a flex? or is it just dated? everyone uses LLVM for all of their compilers.
If you're aware of specific open issues with the project, identified by the founder or others, please share details.
so like why would i ever use this thing terra+lua+"5 guys maintaining it in a basement somewhere" thing?
Los Alamos U.S. National Lab is funding Legion, which uses Terra, https://legion.stanford.edu/overview/
Are the Stanford researchers in a basement? The lab's previous work lead to CUDA. Does that earn them any consideration? How about lanl.gov using the language?I don't know how to explain this to you because you and everyone else around here worships at the alter of academia (especially HYPSM academia) but the answer is absolutely 100% yes. There is literally nothing coming out of any edu lab, even stanford, that has any relevance in this same industry (ML/AI compilers). To wit: there is no company running absolutely anything on top of legion.
> The lab's previous work lead to CUDA
This is like saying a sketch of a car led to the car. No. CUDA is the result of thousands of working engineers years labor, not academia, not academic aspirational hacking.
> How about lanl.gov using the language?
"using the language". using how? every single group at lanl is using the language? they've migrated all of their fortran to using the language? they're starting new graduate programs based on the language? you really have no clue how low the barrier to entry here is - any undergrad can start an association with lanl (or any other national lab) and just start writing code in whatever language they want and suddenly "lanl is using the language". you don't believe? me go look for any number of reports/papers on julia/go/python/javascript/etc/etc/etc coming out of lanl and ornl and jpl and etc. how do i know this? i'm personally on a paper written at fermilab based primarily around task scheduling in JS lolol.
Does Nvidia count?
> This is like saying a sketch of a car led to the car.The ACM Turing Award (computer science equivalent of Nobel Prize) committee believes otherwise.
From SIGGRAPH 2004 slides, Stanford's research on Brook was sponsored by Nvidia, ATI, IBM, Sony, U.S. DARPA and DOE. [3][1] https://web.archive.org/web/20241212055421/https://www.lanl....
[2] https://amturing.acm.org/award_winners/hanrahan_4652251.cfm
[3] https://graphics.stanford.edu/papers/brookgpu/buck.Brook.pdf
I don't understand? To prove that NVIDIA ships legion you sent me a link to a LANL post? How does that make sense? Show me a product page on NVIDIA's domain to prove to me that NVIDIA uses this in a product.
> The ACM Turing Award (computer science equivalent of Nobel Prize) committee believes otherwise
You seem to not get what I'm saying: my firm is position is academia doesn't understand absolutely anything in this area. Zero. Zilch. Nada. And absolutely no one in the industry cares either. So given that position, why is this relevant?
The only thing academia is good for is a talent pool of hard-working, smart people. We take those people and then completely retrain them to do actually useful work instead of research nonsense. The vast majority of PhDs coming from academia to industry (yes even from Stanford) literally are horrible software/hardware engineers. Many of them stay that way. The good ones (at least in so far as they care about having a successful career) learn quickly. That's how you get CUDA, which is a product worth a trillion dollars.
Look I've already told you: you workshop at an alter and you've also clearly never worked at a Stanford or an NVIDIA or a LANL. You'll never be convinced because... well I have no idea why people need mythologies to worship.
Legion is research (where _future_ products originate), not yet a product, https://images.nvidia.com/events/sc15/pdfs/SC5117-future-hpc...
> academia doesn't understand absolutely anything in this area. Zero. Zilch. Nada. And absolutely no one in the industry does either.Would you care to share the names of some software/hardware engineers worthy of emulation by academia and industry?
my drawings of spaceships are also where future spaceships come from. it's plausible right? therefore my drawings of spaceships are valuable right?
> Would you care to share the names of some software/hardware engineers worthy of emulation by academia
sorry that was a mistype - i meant to say no one in the industry cares either. engineers in the industry do understand things because they're working on the things every day.
Are your drawings vetted by DARPA (annual budget of $4B)?
And I would argue that a lot of the staying power of World of Warcraft came from the Lua based addons democratizing the UI. Blizzard co-opted a lot of features from best of breed addons back into the base game. (That and negative pressure in the economy to keep from having every new character wielding the third best sword in the game due to deflation, but that’s a story for a different thread)
I use this for a project of mine where I embed a Lua interpreter where I made several additional Lua modules available.
OBS was one stop along my journey to get there, as there's some Lua and Python scripts that achieve something similar, but not quite exactly what I wanted with multi-screens, so I began trying to modify the script to do what I needed it to do.
In the end I'm going with something called "SparkoCam" which achieves this easily with no hassle, but it costs $40. Since $40 is less than what an hour of my time is worth, it was kind of a no-brainer to abandon the hassles of learning Lua (a many hour detour) and fiddling with OBS Studio to get what I wanted. I have no real use for Lua outside of this, so it would be time wasted on this one project.
There are some places where Lua has surprisingly poor performance. However, in almost everything else, it's one of the fastest around. Luajit is still one of the fastest JITs in existence.
In the real world, rather than constraints of a benchmark, Lua tends to be on-par from what I've seen. When you're not cold-cacheing everything, its memory use is lower. But it's standard library is ISO-C, and that's about it. No batteries. And the others beat it soundly there, whilst being close to the same performance. Which will mean they will just about always win out.
[0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
[1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Now made a little easier for your comparison
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Btw. it seems notable that Lua 5.4.6 is about 40% slower than Lua 5.4.1, see http://software.rochus-keller.ch/are-we-fast-yet_Lua_results....
Here are the results: http://software.rochus-keller.ch/are-we-fast-yet_lua_results...
Overall, the JS/V8 version was twice as fast as the Lua/LuaJIT version. LuaJIT was only faster in the Mandelbrot benchmark. The recent PUC Lua version 5.4.1 is even ten times slower than the V8 version at the time. More recent V8 will likely be even faster. The results also demonstrate that one can get a pretty false impression when just looking at single microbenchmarks.
On the other hand, if the article is saying that it deserves to be more highly rated in the non-embedded space, I think I would need more than a handful of syntactical niceties to explain why it deserves to be in the same conversation as Python, Ruby, etc...
I simply don't see any reason not to use it .. for everything.
I've built games with it. I've built GUI's with it. I've built scientific analysis tools with it. I've used it as a total solution for previously-impossible embedded problems. I've used it for desktop applications and mobile apps. I've automated entire production lines with it. It has huge appeal as a general purpose software development ecosystem - but those who know how to do this, tend to keep it to themselves for some reason.
It can do everything.
There's just nothing I want to do with computers that I can't, legitimately do, with Lua - or its VM. Or, LuaJIT. Or, all of the above and a bit of luarocks built in. Throw some luastatic in while we're at it, bundle it all up as a .deb, and nobody needs to know a thing.
Its just so versatile. I hope someone builds an OS around it some day, making everything and anything the user wants to do, accessible through a Lua module, or some smart libffi wrappers around a handful of .so's, somewhere ..
In my retirement, I'll take antirez' LOAD81, turn it into a proper Lua editor, add integration with luarocks and luastatic and maybe a shit-tonne of IPFS, and then there won't be any need, ever again, to deal with anything else.
I've tried various other Lua debuggers over the years, these are useful:
https://github.com/slembcke/debugger.lua
https://slembcke.github.io/DebuggerLua
You can also use the debug module:
https://www.tutorialspoint.com/lua/lua_debugging.htm
Otherwise, I just use gdb if needed (JIT/FFI stuff) and printf's.
However, I don't really need to debug much these days - more just testing. Its also kind of important to have automated tests and run them frequently with every major change.
Another thing I use frequently in my Lua projects, which I think cuts down a lot of the common bugs that might require debugging, is the lua_enumerable.lua utility module:
https://github.com/mikelovesrobots/lua-enumerable/
Many people love Lua, so I suspect there is a good reason for this.
Its a trivial difference that does not matter.
I thought I read somewhere that Lua was meant for use by non-programmers, so the "end" would be easier to type and read than curly brackets.
Presumably they thought the more verbose syntax would be more readable.
Modula changed it so that all structured constructs implicitly introduced a compound statement, and terminated one where appropriate (e.g. on `else`), but this still required some end marker for the last one.
If the question is why `begin` and `end` and not `{` and `}` - for one thing, Algol was designed to be the language you'd write reference implementations of algorithms in for publication, so readability was more important than terseness. For another, it predates ASCII, and not all charsets in use at the time even had `{` and `}` as symbols.
- }, like C: Lua uses this to terminate tables. You could use the same token for both purposes (Perl5 does) but the reduced redundancy comes at some cost to both readability and syntax error reporting, which are especially important in languages aimed at driveby programmers, like Lua.
- ): Lisps generally just use ) to terminate every construct, reducing redundancy further.
- different end tokens for each construct, like Ada and the Bourne shell (endif/end if/fi): improves readability and error reporting further, but requires more reserved words and is wordier.
- end, like Lua: intermediate among the above choices.
- indentation, like Python: seems to work well enough in Python, and especially helps beginners who are therefore not permitted to indent their code incorrectly, and adds a maximum amount of redundancy for opening and closing constructs (every line is explicitly tagged with its syntactic depth) without reducing readability. However, this aspect of Python was novel when Python was introduced, remains controversial, and has not been widely adopted in new languages.
A funny thing about Lua's syntax is that it's not newline-sensitive, but semicolons are optional. This is valid Lua:
Wherever two things are juxtaposed that can't be part of a single statement, it tries to parse them as two statements. This syntactic looseness means that some syntax errors that would be easily detected in semicolon languages aren't easily detected in Lua. This increases the penalty for removing additional syntactic redundancy.coming from mostly python, i miss a robust stdlib.
As a learning experiment, I was able to use the fennel website to convert my Hammerspoon config from lua to fennel, but I still struggle to add new functionality. Certainly have not delved into macros, but this is where much of the power lies AIUI.
Both Lua and Fennel are tiny languages. You can skim all the documentation in a short evening. Fennel does not even have its own standard library, so keep the Lua documentation close at hand.
I appreciate the simplicity of both Lua and Fennel. There are some ugly parts, but all languages have those.
Janet (later project by Fennel's creator) is a nicer language that is also portable and embeddable, and has a nice standard library, but Fennel runs anywhere Lua runs. Janet is great just on its own for scripting and not sure I would want to use Fennel for that anywhere Janet is available. But as a way to script games in e.g. Löve2D it seems like an excellent choice.
[in defence of raku, while it truly is marmite wrt Ruby / Python, there is a very loyal core of smart people that love the freedom and expressiveness it offers]
FTFY ;) Honestly, I love Raku - it's almost like Racket of the non-sexp world - but the community is microscopic, and the development is slow. It's a shame because, solely based on features and in terms of design, it's a serious contender for the most powerful dynamic language out there. It's packed with interesting features (junctions, phasors, traits, quote/unquote macros (soon), typed variables and subsets, coercible types, augment, OO system with multiple inheritance, interfaces, generics, multi-subs and multi-methods everywhere, custom operators including circumfix, regexes and grammars, etc, etc...) out of the box and it's actually really impressive how all these features work together in a cohesive design.
Unfortunately, the underlying implementation seems a bit chaotic; some features are either not fully implemented or specified, the performance could be better, and so on. So, while I love and still use Raku for almost all my scripting, if I were to start a serious project, I think I'd go with Common Lisp instead. CL has fewer features out of the box, but many of the missing ones are one `ql:quickload` away (or you can use CIEL), while the underlying implementation (SBCL in my case) is diamond-solid.
---
Aside: that `render` method could be sweetened a lot! For example:
or: or finally (using the fact that hyper descends into subarrays): (I think Raku is unrivaled in one-liners!)one liners have their place - but I like to avoid compression for the sake of it
to put some numbers on it there are 146 folks listed on the main IRC chat (plus a bunch more (79) via the Discord channels) right now and 1.5k members of the sub-reddit
Sure! I just thought if we're talking about "Syntactic Syrup of Ipecac" :D we can as well take it to the extreme, just to show how sugary Raku can really be (if you want).
As for the community size, I'm on Discord for a bunch of languages:
Taking your numbers, the community is about as big as GToolkit, which is a niche distribution of a niche language (Pharo Smalltalk). I think that's enough to keep the lights on and even to occasionally produce something really impressive. At the same time, it's not enough to consistently polish all aspects of the language and the ecosystem: with a community of that size, you will always be running into "not implemented yet"/"no bugfix for years" problems. Which is fine in some settings, but you need to keep this in mind.That said, I think there are other warts that can be confusing and make the language difficult. The Nil-Terminated Arrays section points out one oddity about `nil` - but there are many others. There are a number of confusing corner cases (or were - I haven't used it in a while). As an example, if you have a list of values and you pass that list into a new table, the resulting table may be different than if you add each value in that list in order.
I also think that the flexibility of embedding means that you are often learning the Lua <=> C interface as much as you are learning Lua. So jumping from deploy to deploy is more confusing than most languages.
Nevertheless I fully agree the language is underrated!
One time I had a memory leak in a game, something like exactly 4 bytes per frame.
It turned out I was not popping the Lua-C parameter passing stack properly after each frame.
I found this out after some unhelpful person suggested that "all programs just leak memory" lol
What I find fantastic about the stack-based API is that it exposes very few interpreter internals. As a result the authors may drastically change the implementation without breaking the public API. Most importantly, they hide the internal object representation and as a user you only ever deal with plain C types, like integers, floats, C-strings and user data pointers. As a consequence, you don't have to think about memory management because all types are passed by value! In fact, someone could theoretically write a Lua implementation with reference counting and the API wouldn't have to change. It really is a very smart and elegant design.
Any integration code that doesn’t use an ad-hoc helper library looks like assembly listing (not even in disguise, cause it basically is bytecode but in C).
See my other reply: https://news.ycombinator.com/item?id=42519033
> Any integration code that doesn’t use an ad-hoc helper library
There are mature bindings for most languages. You only need to deal with the C API directly when you
1. Create a new language binding
2. Write a Lua extension
3. Call Lua from C
Typically, Lua is embedded as a scripting or configuration language into another project, and in this case you wouldn't even be aware of the C API.
(from elsewhere) It is a low-level API for creating bindings to other languages and as such typically isn't used directly. (Well, unless you are writing C :)
Yeah, unless. But even under this assumption, wrapping handles to values would be much more easier in a language-specific wrapper, because it wouldn’t have to watch out for stack overflows etc and solve the exploding stack problem in general in naive api-user-side loops. This thread started as Lua stack API is better than Python’s, iirc. But Python API already looks like a high-level wrapper doing exactly that, so… what was the point again?
I’m a little concerned that everyone talks high matters here without providing examples or at least looking into “final particles”. Cause that’s where claims get tested against reality. I spent too many years in deep Lua and its mailing list to miss something obvious that everyone knows.
I would agree that the Lua C API itself is not very eegonomic, but the wrappers/bindings it enables absolutely are! For example, integrating Lua in C++ with sol2 is an absolute joy.
I understand how gcs work and their nuances. My only problem was with the "fantastic" part. Lua stack is not "not ergonomic", but absolutely shit to work with, and "there's wrappers" is not an argument, cause they are not a part of distribution but external projects ought to fix the mess.
This would only work when being called from Lua, but not when the host calls into Lua (on the top level).
I do not do a ton of low level development, but my impression is that you do not want a demanding interface. You want an interface that is straightforward, doesn't have complex pre & post conditions, and doesn't crash. Leaking 4 bytes per frame is bad but crashing would be worse! You may think you want a library that refuses to function at all unless its called perfectly, but once you're in the trenches you may change your mind.
To my mind, it's a very good sign when there are popular "adapter" libraries. It means the underlying code (Lua) can be used many ways and people have found best practices for different approaches. When I used it I was not working on a game and it would have been worse for us if Lua's interface was "built for games." Being very bare bones and low level allows flexibility and that flexibility ensures a healthy ecosystem.
I’d like to see a specific example (low-key, I left Lua) on how its stack provides more flexibility or anything, because in my humble experience it does not, absolutely, do anything like it. It just makes you play these off-by-one games all day for no good reason.
You can, ofc, write your Lua wrapper like your example - all in one place. It reduces the risk of misuse. But you could also do it differently - which is the flexibility that I think is a strength.
It’s ok, but should be much better.
Once upon a time Supreme Commander used Lua. Perf on consoles for SupCom2 was catastrophically bad. We switched to Kore which was effectively Lua but way faster (10x?).
Kore was bought by Havok and rebranded into Havok Script. Havok got bought by Microsoft. I’m honestly not even sure if you can still buy it today. It’s not listed on their website.
These days I think you’d use Luau? I’m not sure. https://github.com/luau-lang/luau
It has a massive advantage in the existing ecosystem. People don’t always love it, but millions of them already know it. You don’t need to re-learn array indexing and nil termination quirks and the dozens of other things where Lua is subtly different in ways that you’ll find out at runtime.
There are JS engines in all sizes, from tiny interpreters to world-leading JIT performance monsters. And most operating systems nowadays ship with a high-quality engine and embedding framework that you can access without installing anything, like JavaScriptCore on the Apple platforms.
Do your users a favor and just use the boring standard language instead of making them learn a completely new one that doesn’t offer meaningful practical benefits for them. It’s probably not as much fun for you, but you’ll save time for both the users and yourself in the end.
Anyway, every language has some fans, and I don't mean to dismiss them, but ubiquity is not the top priority, all the time.
It is beyond a footgun, more like a footgun concealed as a shoehorn.
Either way I find embedding a Lua VM into a project so much easier than embedding a Javascript one.
Noobs are going to find the footguns in any language. Every language can be misused. I've been using Javascript over 20 years and have never thought it was any worse than any other language. Yes, it's dynamic, and yes, you can do stupid things with that dynamism, but for someone who actually knows how to use it, it's quite easy to work with.
I recently started using Lua and I'm not impressed at all. It's just another language, it's really up to the programmer what can be done with it, just like Javascript.
If this is true, why not write everything in C++?
Javscript includes so many footguns that people know are bad (see the billion of examples of different types being added and getting random answers). Every langauge has some difficult parts, pretending that means any amount is fine is crazy. Javascript has earned its reputation.
Almost every "footgun" people cite about Javascript is something that would never be written as working code, by any competent programmer - they are "WTF" in that the person writing the "gotcha" has to go way outside of normal programming behavior to get the "WTF" result. Most of these "WTF" examples should be taken with a grain of salt.
If you're expecting to "WTF" about Javascript's type inference/coercion, that works according to some logical rules.
https://media.geeksforgeeks.org/wp-content/uploads/advanced-...
This matrix should be pretty easy to understand. Yes some things might seem weird like [[]] == false, and [1] == true, but nobody should be writing code like that, it's just illogical to do anything this way and I don't consider it to be a "gotcha". "Just because you can, doesn't mean you should" can be said about footguns in every language.
Yes, there are "WTF"s, but just like if I were to write a C++ "WTF", it wouldn't be something that most people would be tripped up by unless they were specifically looking for a way to say "WTF" about the language.
These "javascript sucks" internet arguments have persisted for decades, and it's always complaints about the language that are citing unreasonable examples to prove a point that a programming language has some rough edges. Every programming language has some rough edges.
Javascript maybe has more footguns than some other languages because it is so dynamic, and the dynamism has its benefits too that other languages don't have. I do like it and I use it effectively, YMMV. People just seem to be specifically mad at Javascript because it's so popular and was made so by being the defacto standard for web browsers. It's part jealousy and part frustration that their favorite programming language isn't the one that web browsers include by default.
For better or worse, JavaScript is the Lingua Franca of modern programming - you have a high-performance runtime environment pre-installed on basically anything, and readily installed on greybeard boxen if you are that way inclined . It is my "go to" for all hobby projects, frontend and backend. So versatile and a genuine joy to code in.
In any reasonable language, even a dynamically typed one, this should just be a type error because it's not a Boolean, but JavaScript decides to do something almost certainly unwanted. This is the real source of the complaints, not the fact that you can write “[[]]==false” on the console and laugh at the results.
That isn't "moderately complex code", that's completely stupid code and you deserve the grief if you are willy-nilly setting a boolean as an array. Maybe learning the difference between a bool and an array is something you should have done earlier in your programming journey.
Also love how you assume I've never worked on "moderately complex code" as a slight on me, when I've been writing software for over 40 years - just because I don't do stupid shit like assigning an array to a boolean variable does not i dicate I've "never worked on moderately complex code". That's some kind of gatekeeping troll.
>any reasonable language
I'm really not sure how stupid you have to be to assign an array to isAuthenticated. No, it is not something I have ever done in any language, because I'm not an idiot. YMMV.
> This is the real source of the complaints,
The complaints come from people doing stupid shit to themselves. It was easily avoidable and they got mad at the language?? Should we blame the language for every stupid thing people do with it? Will you also blame English for letting you write an obviously stupid comment?
Js needs no forwards which in Lua turn into ugly “local foo; … … … foo = function()…end”. All functions are “hoisted” by default.
Js also uses “export”s or “module.exports[…]” instead of returning a symbol table, which allows cross-referencing.
Js has a function scope (var) together with local scoping.
Js doesn’t allow accidental shadowing in the same scope. Lua: local x; local x; silently creates two variables, good luck debugging that closure.
Js type coercion is only insane in the parts where operation itself makes zero sense. Please don’t tell me that ![0] or [] + {} makes sense to you or that you use them in other languages. Or that s+n and n+s seems safe everywhere except in js.
Js has a ternary operator and ?? ?. ??= operator group. Lua only has that “and-or” crutch and “not X” which returns true only for nil and false.
Js “…arr” is a first class citizen while in Lua “…” decays into its last value if you want to follow it.
Js (normally) ordered properties system is much more convenient in dev and debug time than an unordered mess of meta-hashtables.
Js doesn’t require you to
which is extremely abhorrent DX in 2024.Javascript took an extremely high-complexity, high-cost, and nearly-all-effort-in-just-two-impls-for-over-a-decade route to get to the current (stunning) performance though. How do small-scale embedded interpreters stack up? That's what many things that aren't Electron apps will be using.
I ask this honestly, to be clear - I'm not sure how they stack up. I know Lua has historically targeted and dominated this area so I'd expect it to win, but I have not seen any data and I largely agree - JS is a massively better development environment now, and I would love to choose it. Everything I can find pits it against v8, and that's kinda like comparing the performance of a bike against a train - they are so different in many situations that it's meaningless. Otherwise I'd probably choose WASM today, or maybe docker containers where performance and platform needs isn't an issue.
Moddable's XS runtime is a wonder, with 99%+ ES2023 conformance even on devices running at 80 kHz with 45 KB of free memory. https://github.com/Moddable-OpenSource/moddable
QuickJS has somewhat higher requirements, but also a great candidate for embeddable mobile and desktop applications, and is well-regarded. https://bellard.org/quickjs/
Lua as an embedded language already had over a decade head start.
Microsoft also had server-side JS, and still do. Active Server Pages (ASP) were scriptable with javascript since 1996. Microsoft's JScript.net has been around since 2002 (even if not well supported or maintained by M$, it still works). I've written web back-ends, Windows desktop applications, and even .dlls with JScript.net as far back as 2003.
Javascript has been Adobe's scripting engine for Photoshop since 2005. Sony chose Javascript for Vegas Video's scripting language in 2003.
Javascript has so many uses outside of a web browser, and has for a very long time.
But I guess my anecdotal experience doesn't matter, because I wasn't a "mainstream" developer? From my point of view, everyone else was missing out on using one language for front-end and back-end.
Context switching has a real cost in terms of developer burnout, and it's something I've happily avoided for 25 years.
Also there are whole books written on the quirks you need to master with Javascript. Lua has its rough edges but it's a much simpler language, so there are far fewer of them.
Game dev, possibly (even that seems highly debatable). But the article was making the case that Lua should be more widely used outside of that niche.
And a lot of the issues with JavaScript just wouldn't matter for an "embedded scripting engine" use case imo.
I also think 9 out of 10 languages are overrated and we have far too many.
* Duktape python tooling is obsolete, migrate to JS-based tooling! *
JS and Lua are very similar feature-wise and I imagine Lua hasn't gotten the massive JIT runtime optimizations that JS has, so JS seems a far better choice these days.
The fat better arguments are that it's more common and there are more libraries.
So if you only need a small language to do a couple thousand lines of code in than it still might be a better choice. But for larger projects JS still seems better due to the ecosystem and knowledge available.
But if you compare some games like baldurs gate 2 where most of the event logic and quest state is in the scripting language, then maybe it is worth spending the time to add JS. But if it is like Civilization 6 where only small stuff like UI and I/O mapping is in the scripting language then it is a different matter.
I'm no developer, I do Linux and some bash and Python, and Lua looks "familiar" to me, so I can quickly pull up something simple.
With Javascript everything looks so alien to me, and it's a pity because there is lots of things that run Javascript (functions in the cloud and the like).
This is not the case for Javascript, which has far more syntax and many duplicated subtly-different features (var/let, class/plain objects, this definition depending on lambda syntax). Not something easily grasped by a new user.
And sure, there are many Javascript snippets floating around and its possible to pick up some subset of Javascript by osmosis. However, this approach leads to a "poke it with a stick until it works, while understanding nothing and slowly going mad" UX from what I've seen. Which is not something I would aim for in the kind of productivity tools embedded scripting languages are used in.-- I find these tools are far more satisfying and productive to use when you have a simple mental model which clicks in place after a bit of effort up front, letting you work after without having to consult docs for every step.
(Of course this is not an issue if you're targetting only developers proficient with Javascript as users, but this is not the case for most applications outside of the webdev space.)
The most fun was when I made a game engine ("engine") in C++ that used Lua for all the game logic. The fast debug cycles are very useful there. Maybe I should still do that for Rust gamedev, since Rust notoriously compiles slowly.
The second-most fun was an IRC bot that could reload its Lua logic without disconnecting. Nowadays I'd write a bouncer and just let the main process restart. (Or use wasm, a wasm IRC bot would be good...)
The third-most fun was using Lua as a Turing-complete configuration language. If I needed an array of something, I just made Lua make an array, instead of generating Lua code.
One of the least fun was the Blender plugin that exported 3D models as generated Lua. If I did that one again today, I'd just use MsgPack or some ad-hoc binary format.
It's been a while since I wrote any Lua but I didn't think it was intuitive at all. Why are arrays tables, why are they nil terminated and why do they start at 1.
I don't think it's underrated either. It's very popular for writing scripts in (older) game engines.
I might be able to provide at least some insight into that.
One of the things I found while implementing my language was that the ordering of elements in a hash table is essentially random. Literally random if you mix in some randomness into the hash function's initial state in order to harden it against collision attacks.
Many languages guarantee insertion ordering. It's a useful property to have, makes testing easy and deterministic and it's intuitive for users. A simple way to implement that is to put a perfectly normal array of values into the hash table, and then have the buckets which the keys hash into contain indexes into that array. That way the indexes of values remain stable no matter what the keys hash to.
So a sufficiently advanced hash table will have an array of values inside it. I suppose some language designers took one look at that situation and decided to just merge both types already by using hash tables for everything. For example, PHP also did that. Associative arrays, they call it.
"Associative array" is just the term used for hash tables (dicts) in Awk, bash, Tcl, and, most relevantly for PHP, Perl4. It doesn't specifically mean that the language lacks a separate container type for small-integer-indexed arrays. It just means that that particular container type can be indexed by something other than small integers, usually strings.
It doesn’t mean HTs are the arrays which should be used to store seqint-indexed values.
In case anyone is curious about it:
https://mail.python.org/pipermail/python-dev/2012-December/1...
> In addition to space savings, the new memory layout makes iteration faster.
> Currently, keys(), values, anditems() loop over the sparse table, skipping-over free slots in the hash table.
> Now, keys/values/items can loop directly over the dense table, using fewer memory accesses.
The "dense table" is the array of values. The "sparse table" is the actual hash table, only the values index into the values array.
I thought the insertion ordering was implemented at around the same time as a result of it. The data structure ensures it pretty much for free, seemed like the obvious next step. It appears I was mistaken about that.
But I fail to see the logic behind seeing this and merging array/table ideas. Feels like one of these irrational decisions you make in a dream.
And Lua doesn’t use hash table to store arrays. Its tables have two different parts (hash, indexed array) with foggy distribution logic which reflects into “# vs ipairs” issue. This merge is purely synthetic and doesn’t project any underlying implementation idea.
Looking from today's perspective, I also fail to see the logic behind it: separating arrays and dictionaries does not make your implementation more complicated than having a single backing data structure for both and taking care of all the special cases that arise from such a monstrosity.
The best explanation I can think of is that this was just a fad nobody questioned too much. Back in the 1980s and 1990s, there was a strong distinction between "scripting languages" and general-purpose dynamic languages (like Lisp and Smalltalk). Scripting languages were supposed to be used to write small scripts to automate stuff. Type-safety was not something that any mainstream language considered back then (it was a thing in niche languages like Ada, SML and Eiffel), and speed was not a concern for scripting languages.
A common philosophy (though I never seen it clearly stated) for scripting languages seems to have been to "be liberal and try to work with whatever the programmer does".
So you shouldn't require to declare variables in advance (that's a lot of hassle!) or throw an error if a function is called with the wrong number of arguments.
And what if the programmer tries to add an integer and a number written as string? Just coerce them. Or better yet, make everything a string like Tcl does. Or at the very least, don't be a spoilsport and have separate types for integers and floating point numbers!
Ok, so what if the programmer tries to use a string index? We cannot coerce non-numeric strings! Well, why don't we just unify arrays and dictionaries like we've just unified all our number types? Now you don't have to worry about invalid indexes since all indexes are converted to strings! And if the programmers want to use 0-based, 1-based, 2-based or 42-based indexes they can do whatever they want! They can even have non-contiguous arrays or mix arrays with associative dictionaries. Total freedom.
This thought process is completely imaginary. I suspect not a lot of thought was given to this type of flexibility at all back in the day. It was just a widely accepted assumption that scripting languages should be flexible, free and relaxed. And merging arrays and dictionaries (or "all arrays are associative arrays" as it was often called back then) was just one more idea that naturally fit into the zeitgeist, like type coercion and "assignment to an unknown variable sets a global variable by default".
So tldr: it was a fad.
Not to my understanding. What is called "array" in the language is indeed associative, but lists (heterogeneous vectors, like Python's) have always been here and distinct.
But otherwise, insightful post. Awk is also the first thing that came to mind, with its funky arrays and "undefined variables are initialized on first use to the empty string or 0 depending on the context".
For the same reason why (standard) arrays are associative in PHP and JS, as well - because, from a certain perspective, a regular array is just an associated array with consecutive integers as keys, so if you're trying to keep things simple, it seems like a straightforward unification.
> why are they nil terminated
It follows from them being associated arrays, and Lua's pervasive use of "nil" meaning "this thing is missing".
> why do they start at 1
Because humans generally start counting from 1.
I’m not parent, but for me the question is not literally why did they choose 1-based indexing. That’s easy to answer. The question is really: why haven’t they realized by now that the disadvantages of 1-based indexing far outweigh the benefits when it comes to programming, and changed it? Backward compatibility is a valid reason, but lots of other people have tried 1-based arrays and reverted to 0-based once it becomes clear that 1-based arrays causes more problems than it solves. Even humans who count starting from one will soon want to use 0-based the moment they need to add two array indexes together; we don’t want to have to start from 2 in that case, right? Early versions of the famous Numerical Recipes in C started with 1-based arrays, but removed them, and got slightly better. :P
Exactly, it seems popular in places where the alternative is even less usable (e.g. write your own C++ plugin or thing). For that it's fine.
For example, there is a lua plugin for x-plane and it's great. I've done some minor things with it even. But that didn't make me a fan of the language. It's fine for simple things. But a bit limited.
The x-plane API is pretty low level and written in C++. So something that sits on top and is a bit more accessible is great. There are a lot of lua scripts available that you can run in there that are pretty nice. But the more complicated ones get pretty hairy quickly. Makes you long for something more structured.
I don't have the time, but I did consider working on adding a websocket API at some point to make it easier to integrate with from e.g. a web page or some kind of standalone process. I think they are now planning to add that feature themselves so there's no need to work on this anymore. And they have a low level UDP API currently that exposes all sorts of things. But a websocket API with some good client libraries for different languages would make a lot of sense.
[1] https://github.com/luau-lang/luau
[2] https://github.com/luau-lang/rfcs/tree/master/docs
Its greatest improvement over vanilla Lua is better sandboxing. Lua is just pain in the ass to embed safely. So many footguns.
This blog has just three posts, none with significant content. I was curious how this article even wound up on HN, until I realized that it was submitted by the blog's author: https://news.ycombinator.com/user?id=nflatrea
I wrote this post https://andregarzia.com/2021/01/lua-a-misunderstood-language... some time ago and it kinda touches similar points as the OP. Read on if any of yous want to see yet another person talking about Lua.
But seriously, they do have some interesting quirks which make sense if you think of it (and maybe Lua as a semi-contemporary got its ideas from there), specifically, hashes are pairs of keys (even/odd) in a flat array... and of course objects are just key/values, but with some values being names and functions...
https://perlmaven.com/creating-hash-from-an-array
I think it's clear in retrospect that it makes more sense to keep lists and maps separate even in a high-level dynamic language that doesn't care much about perf, and any purported gains in simplicity from having a single data structure are illusory because of footguns. But perhaps it wasn't so clear before we had enough experience with this kind of design.
Then again, Python had a clean separation between lists and dicts from the get go, and it predates Lua as well as PHP and JS...
Lua is mostly used inside games, and the JIT compilers for it are said to be pretty good. (I haven't tried this yet, but will have to soon.)
But yeah, my point is that one of the most common arguments against using associative arrays as the data structure is that perf overhead is unacceptably high if you use it as a list, but we've learned how to implement it in a way that can adapt to this use case at runtime just fine a loooong time ago. But regardless of perf, it just makes more sense to keep sequences and maps separate, not because of any specific technical reason, but because it's easier for humans to reason about things and model them that way.
No first class support for OOP, you got to use tables instead.
No "continue" keyword to go to the next iteration of a loop. So you have to use some ugly elseif.
No try catch!
It's a dynamic language so everything goes. A function will return nil or a number or whatever you wish. A function can also returns any number of variables, so obviously some moron will abuse it: looking at you Blizzard intern (you must have been an intern right?) that thought returning 20 variables on the combat event log was a good idea. Ever heard of tables???
The LSP can't do miracles when there are almost no rules so auto-completion is more miss than hit, which is quite shocking in 2024. The IA is helpful sometimes but create subtle bugs some other times. A simple uncaught typo will create a hard to find bug (yesterday copilot auto-completed myObject.Id instead of myObject.id, there went 20 minutes of my life trying to find why some code that was running solidly for weeks was now failing silently).
So all in all, Lua is fine to write small imperative scripts of a few hundred loc. Anything bigger and more complex you should run away IMHO.
I never realized C# was such a well designed language until I tried Lua.
I think it just doesn't have much competition in the "embeddable languages" space unfortunately.
As another commenter said, JavaScript is probably the best choice in most cases today.
There's a list of options here but most of them are not really good options usually:
https://github.com/dbohdan/embedded-scripting-languages
E.g. Python would be a terrible choice in most cases. Awk? Some of them aren't even general purpose programming languages like Dhall.
If you narrow it down to embeddable-like-Lua there are hardly any.
You get meta tables instead which allow you to trivially write an OOP system that perfectly fits your needs. Yes there is more than class-based OOP.
And I mean trivially: https://lua-users.org/wiki/SimpleLuaClasses
For some people not being locked into one form of OOP or any OOP at all is a huge advantage.
> No "continue" keyword to go to the next iteration of a loop. So you have to use some ugly elseif.
Yeah, that sucks. Thankfully Luau fixes that and basically all annoyances I have with Lua. https://luau.org/syntax
> I never realized C# was such a well designed language until I tried Lua.
Lua and C# have fundamentally different design goals. C# sucks for embedding, you have to include a huge runtime. Godot still hasn't figured out how to support C# on web builds. Lua values minimalism and flexibility. That obviously means it lacks some quality of life features of bigger general purpose languages.
Language design is about trade offs.
Not exactly mainstream, but so simple and elegant.
[0] https://redbean.dev/
[1] https://github.com/pkulchenko/fullmoon