Show HN: I rewrote the 1990's LambdaMOO server (github.com)
I've always wanted to see this kind of thing modernized and further developed. Over the last 25 years or so I've worked on similar but novel & improved things, but never finished.
So I decided to just re-implement LambdaMOO and use that as a base, instead and keep compatibility as a goal, but build it out on a more modern foundation that takes advantage of multiple core machines, newer network protocols, newer connectivity methods, uses MVCC transactions for the shared database etc.
LambdaMOO is a somewhat extensive system in that it is composed of compiler, a virtual machine, an object database, user permissions system, network runtime. In some ways it's kind of like a shared, text-based Smalltalk image/runtime... So quite a bit to implement and get right before it all works together.
The big challenge throughout has been slavishly maintaining backwards compatibility so existing "cores" (databases) work.
It's not done, but it's darn close. Would like for people who are into this kind of thing to check it out, and maybe even help.
Many of the technical aspects here are still provisional, but this is the start. Constructive assistance welcome.
(Yes, it's a rewrite in Rust, but that's not really the point, even though that's a cliche that's fun.)
70 comments
[ 3.1 ms ] story [ 332 ms ] threadI have some questions:
But sticking with compatibility has allowed me to enforce development discipline, basically. And then I'll move it onwards from there. I have many ideas. And a product concept (if I could find an investor...). But I felt it important to start with a thing with existing foundation and use cases before doing the exciting stuff/changes.
Re: world state / transactions -- yeah, basically all I/O and mutations happen in a transactional context, and then at commit time conflicts are resolved; if they're not resolve-able, the transaction is retried in a new state. As for overhead, yes potentially maybe a lot, but it's also a solvable problem; this is how an MVCC SQL database (like, even Postgres) works. TLDR it's likely inefficient now, but I believe I can make it efficient. And I think it's the best way to solve the shared world state problem and still meet user's expectations of consistency. Original LambdaMOO had a global interpreter lock and only one user was mutating the world at a time, in their given tick slice. Not gonna scale.
Re: the MOO client, it's `rmoo.el`: https://github.com/lisdude/rmoo -- it's been around for a long time (25, 30 years?) and it and/or MOO.el (another emacs one) are how/why I learned emacs in the first place. I had to minor patch my local copy to make it work with emacs 29.1, though I don't know elisp well enough to say whether my patch is the right thing to do or not, so...
EDIT: I just re-read and saw you asked whether each new verb call creates a new state, and actually it's not that. It's each new command or network transaction, which can in fact lead to a chain of verb (method) calls. Recall a MUD has a command loop, so basically a transaction begins at command entry, and commits when all the invoked methods/verbs provoked from that complete.
I’ve previously seen (abandoned) attempts to convert the Erlang runtime directly into a MOO runtime. Each command being a concurrent actor-process would make a lot of sense to me. (Though a semantics where all commands are MVCC against the at-inception world-state and will rollback or retry on conflict, kind of moots having any kind the advantage of a runtime based on message-passing, since you’re not going to be passing messages between separate concurrent “sessions.”)
In the end a MOO is just a database with fancy stored procedures. And the original code is not a particular... good one. Handling serialization conflicts is what databases have to do as one of their core abilities, and LambdaMOO just tries to punt on the whole question.
I have attempted similar systems in C++ before, and got fairly far along at times. But Rust really solves a lot of problems, I tear my hear out a lot less.
This is a "systems" type of application; this is a database and compiler and a virtual machine and language runtime. I'm not super concerned with optimization at this point, but I will likely have to be at some point. Being closer to the metal is key for me on a project like this.
I'm likely to e.g. rip out RocksDB and replace it with my own database engine at some point (something I work on here and there as another side project, unpublished still.) I will also likely need a better garbage collection solution. With Rust I have options.
The community of libraries available for Rust is good. The right tools are there. And the language is expressive enough to make it work well.
That said, I have mulled writing the "host" portion -- the layer that manages network sockets and web endpoints, in Elixir/Erlang:
The 'daemon' portion of the system sits behind an RPC layer built around ZeroMQ -- so that layer can be restarted/upgraded without losing active network connections. Or can sit on a separate machine for scalability reasons, etc. So it's entirely possible to split what I'm calling the "host" (network facing layer, what we used to call a "frontend" when I worked at Google, but most people think of that term as referring to a UI layer) from the backend "daemon."
Anyways for me this project ties together a bunch of interests in systems programming I've had for years. It becomes a kind of synthesis project.
I've tossed around doing MUDs in Elixir before. I'm not familiar with MOO though.
State has to be owned by a process - who owns the state of the world? Process per "thing" is too fine grained - you're going to have race conditions and deadlocks. Process per "room" can work, but it's still not easy. There are still some cross-room coordinating actions you need to tighten up, and this makes general scripting very hard.
Keep in mind, when data is passed "between" processes it is deep copied. This can be a big problem depending upon your process model and how many things your players can carry around.
As an experiment I made my own software transactional memory. It worked, assuming no network partitions, but designing and testing it to work in an environment with those was beyond me.
BFT STM is a noble goal and, tbh, building performant fault tolerant atomic transaction systems outside of C++ must be quite challenging with all the abstractions atop it
The Erlang runtime has other stateful things besides processes, and manipulating data within these and/or getting a handle on data in these places doesn't necessarily involve copying.
• ETS tables, which hold state privately to themselves (with copying in/out) in a similar way to how processes do. If you manipulate data inside ETS tables by "sending compute to data" (think Redis INCR) rather than by "sending data to compute", then no copying happens.
• "Globals" in the form of data compiled into read-only versioned modules loaded into the modules table and available to be referenced from any/all active processes, only copied if the module gets unloaded before the process dies. (Originally this was just a design pattern — https://github.com/discord/fastglobal — but it eventually became its own runtime feature in Erlang 22, https://www.erlang.org/doc/man/persistent_term).
• Large binaries (anything over 64 bytes) aren't allocated in an actor's memory arena, but instead are allocated in a special global-per-node binaries heap, and then ref-counted, where each actor-process holds one reference to each large-binary it's using, and then each read-only slice of that binary, in turn holds a reference to the per-actor reference handle for the binary. One clever technique for sharing a large "database" of data between many actors, is to store the data encoded in a large binary in an encoding that is efficient to partwise-decode; and have the "lookup" operation just parse+decode the appropriate data out of the binary. (This is how erlang:module_info/2 used to work — the global modules-table itself holding a set of references to the loaded modules' binaries, which module_info would then parse metadata out of on demand.)
But yes one approach could be to take something like Postgres and write an alternative stored procedure implementation, and give it access directly to the query execution later bypassing the SQL parser maybe.
telnet lambda.moo.mud.org 8888
There's Connect 4, Scrabble, Monopoly, backgammon, etc. Even Twister.
Pavel Curtis is still out there, LinkedIn says he recently left his job at Microsoft after 20 years. He also runs a puzzle shop: http://www.pavelspuzzles.com/site/
Check out blightmud. It's kind of tinyfuge-the-next-generation.
There's a nice collection of MOO resources: https://lisdude.com/moo/
It is too bad some of the old MOO dbs like ScrabbleMOO and NYC MOO are lost to history...
http://mushclient.com/downloads/dlmushclient.htm
Looks like the VMoo website is still there but is having... issues. But this is the last release. If you have a high DPI display you may find it slightly lacking, even with Windows compatibility settings applied. Which is a shame because VMoo was always the best client.
Most GUI-users use Mudlet (https://www.mudlet.org/) these days. Unfortunately, as far as I know, it lacks proper local editing support. If you're feeling adventurous you can work around that with something like MUDMixer (https://github.com/tms88/mudmixer) to proxy local editing.
My son came to me (after he ran out of his time limit on Baldur's Gate 3) and asked me what I was working on. I didn't know how to explain it. Or show it. :-)
You can still get help from the author through the forum/discord, when you're running into something confusing
I never really go hard w/ it, merely being able to version your aliases/triggers/etc is nice!
https://github.com/search?q=owner%3Ashmup+tintin&type=reposi...
https://github.com/ingwarsw/tinyfugue
-adrian / erikarn
LambdaMOO Takes a New Direction (1992) - https://news.ycombinator.com/item?id=35856654 - May 2023 (15 comments)
LambdaMOO takes a new direction (1992) - https://news.ycombinator.com/item?id=22680965 - March 2020 (29 comments)
Lambda MOO Programming - https://news.ycombinator.com/item?id=20405167 - July 2019 (1 comment)
Lambda MOO Programming Resources - https://news.ycombinator.com/item?id=16327975 - Feb 2018 (1 comment)
Exploring 3-Move – A LambdaMOO inspired environment - https://news.ycombinator.com/item?id=14075439 - April 2017 (15 comments)
---
The "new direction" article was made famous by Clay Shirky in "A group is its own worst enemy". I won't list those here since it's a generic tangent but there are past threads at https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que....
https://www.villagevoice.com/a-rape-in-cyberspace/
It looks like it's been submitted to HN several times, but has never gotten many upvotes.
cmrdporcupine 18 hours ago:
> "I was there, Gandalf, 3000 years ago"
> From "A Rape In Cyberspace: "no hate, no anger, no interest at all. Just…watching. Others were more actively unfriendly. “Asshole,” spat Karl Porcupine, “creep.” But the harshest of the MOO’.. "
> I was Karl Porcupine. Quite young then, though.
I was however, friends with legba at the time.
I also found this project randomly on github at one point, not sure how far along it got to being usable: https://github.com/verement/etamoo
If I'd seen that a year ago, I'm not sure I would have started. Hard to say.
I admire Haskell, but I could never really grok it.
Does it use async I/O / is it one thread still?
You might find this conversation interesting about multi-threading in LambdaMOO: https://groups.google.com/g/moo-talk/c/omF68ZM9rZc/m/B3f-jj4...
There's still ticks and time limits, but they're there to impose time / resource limits, not used to control access to shared state.
Tick-based time-slicing to the database is one reason why LambdaMOO (and its offshoots like toaststunt, etc.) has intrinsic scalability limits. (In the 90s, LambdaMOO-the-actual-MOO used to lag like a son of a bitch under load...)
(This is not to say the actual impl of MVCC I have right now is ideal. It's a work in progress. I just defer to RocksDB's conflict resolution, and it is built on assumptions of "single writer, multiple readers" at its foundation, which isn't ideal for what I'm doing... )
I've seen MOOs that had verbs for performing HTTP requests, through shelling out to curl IIRC. If you have a verb that reads a value, sends it out and updates it based on the response, what do you do if the value changes in the meantime? When is the transaction committed exactly?
The transaction gets committed at the end of command execution, and then all the I/O that "occurred" in the narrative log gets output to its destinations. If a serialization conflict occurred, the output buffers (and all other mutations) are discarded, and everything retried, again. (In theory. Right now it just rolls back and barfs, because I have not finished this layer yet :-) )
Where this falls over is with MOO's synchronous "read" builtin, which is used in places for e.g. password changes or prompts. This is something I have not tackled yet, but will probably just have to deprecate, and will be one of the only things breaking compatibility with existing cores.
Alternatively I could or might provide a transaction mode which is "softer" for tasks that need that kind of dependent input.
It will require some thought. But on the whole MOO's model is similar to a database: In a DB it's query in, commit, see results; in my system in it's command parse, execute verb, commit, flush I/O results out.
For example:
color = player:choose("Pick a color", {"Red", "Orange", ...});
"read" def presents a problem for the transactional model. So like I said I have basically 3 choices: not implement it, or loosen the transactional constraints for verbs that require it. the latter would be something like: as a user I accept that this could blow up at commit and require me to enter the prompt again ("I'm sorry, something went wrong and I'll need you to enter that again."), or treat it like suspend() which actually creates a new transaction (commits and then creates a new one)
This is what MOO was effectively mostly doing already, through a big ol' single threaded interpreter we all took turns on. Only one task was running at a time (sort of, you could suspend() and return later, and then all bets were off on consistency). It's just... that doesn't really scale too well.
I hadn't looked at doing this in Rust yet, so it'll definitely be interesting to give the code a read, particularly with the focus toward modernization.
I've had similar thoughts, so let me say "Congrats, this is awesome"! It definitely warms the cockles of my heart to see work like this. I just wish I had the free cycles to work on something similar (or even contribute to your project). But there are too many demands on my time right now. :-(
From "A Rape In Cyberspace: "no hate, no anger, no interest at all. Just…watching. Others were more actively unfriendly. “Asshole,” spat Karl Porcupine, “creep.” But the harshest of the MOO’.. "
I was Karl Porcupine. Quite young then, though.
Just the other day a friend went to a concert, Mr. Bungle, and so ended up on wrong wiki: https://en.wikipedia.org/wiki/A_Rape_in_Cyberspace
So I went down that rabbit hole, read the essay, bought the book, and days later here appears Karl Porcupine
I've also wanted to give each object an isolated environment (make each one an actor), but I haven't thought too deeply about how exactly that would work and look. I've been planning in the back of my head to do this at some point in the next year, and was planning on implementing on top of the original codebase, but if there's a rust version, I'll use that.
I know it wouldn't be compatible with existing cores, but I'd be using it as the backend for a graphical ui anyway.
How did you go about converting the code? Did you just read through it, or did you start with a c to rust tool? I always thought the programmers manual and codebase were quite concise and readable.
One last thing, there were patches in the 90s to add journaling on top of the checkpoint system... Did those ever get merged into the main codebase?
Thanks for doing this! I'm excited!
Websockets and utf8 as well. Lovely.
Mixture of approaches. No automatic translation. After bringing over the AST and opcodes, etc most of it I wrote without reference to the original code, with the functionality in mind. Where MOO was doing something a bit funky, I often wrote code with a second window open to look at how they were doing it. But in general it came down to running a core over and over again and finding how it broke. MOO has some unexpected behaviours in spots that grew over time. If I were to start over with this, I'd invest the energy in ripping out parts of MOO and turning them into a library and then write tests that run them side by side through the same opcode sequence and compare output. But MOO was built quite monolithic, so that's hard.
Re: maps, MOO's value types are immutable/copy-on-write. So while I want a map type, a typical HashMap may not make a lot of sense, as it would be doing a deep copy on each addition. Some kind of functional persistent map would probably be appropriate. Syntactical sugar for maps would be nice, and I will probably add it later. But I'm also not as concerned with the MOO language itself, as I see it as only one language among potentially many to run on the system.
"I've also wanted to give each object an isolated environment (make each one an actor)"
Maybe related; I intend on reworking the object reference system such that object numbers and $sysobj-references are a form of published capabilities. That is, I will be introducing "private" objects with some sort of signing & publishing & capability style references, and slowly deprecating object numbers. But that's all post 1.0 stuff.
Very fun. I expect the parallelism will also be strikingly better with a modern design and implementation!
Very nice!
Also don't care to have 'lambda' in the name, since the point here is this is really a net-new impl of the MOO runtime not derived/forked from the LambdaMOO sources (though compatible, at least to start.)
In fact MOO itself is also not necessarily something to emphasize because because the idea (after 1.0) is to support other language runtimes to run in the same shared persistent authoring environment. MOO support is just the initial 'hook'.
And I wouldn't emphasize the OO thing either. My intent is to explore a more logic/relational oriented model going forward. And "OO" kinda means something different now than it did back then when MOO started before things ossifying into the class-based dispatch model of Java/C++ etc.
It might just be I need something entirely arbitrary and brand-ish.
Naming things is hard.
One of the bigger challenges with the web UI is that HTTPS isn't really optional anymore, and the enCore database has a ton of `http://` links. So it requires a reverse proxy in front that can rewrite those, luckily in Nginx it's possible to use a small Javascript program to do it.
A while back I wrote a MU*-style system in Ruby, using js as the scripting language. It has a web based tool that gives object owners a URL for each object, which lets them edit the attributes and code in a better way than just line oriented editing. It also lets external services call back to that object to trigger actions.
I never did anything much with it though.