This is an interesting idea. I was thinking about something similar in Smalltalk or Erlang the other day. Mostly building on the actor/object+mailbox concept.
I didn’t actually do anything with that idea yet but may look at the idea in Elixir this weekend.
I was thinking about Smalltalk as well before I made Autolith. I ended up going with Common Lisp because I know Lisp much better (last time I used smalltalk was like, whew, 2014 or so) and because it has better platform support and ecosystem (at least in my experience).
I think Elixir could be great, I knew a guy who was trying to do an agent in Elixir, but sadly didn't get far.
Keep me posted if you get anywhere! And if you'd like to try Autolith, I am happy to help with issues/questions on our Zulip, haha
It's a fun thing to do. I'm really intrigued by the idea of a harness that can tune itself to a particular project. As I've been hacking on samizdat, I realized that you want to have a set of canned workflows as a starter pack. Then the tool makes a copy of that in each project, and keeps adjusting the workflows based on where it gets stuck, or when it thinks of ways to do stuff better. So, each project can evolve in its own way.
The key trick is that workflows are represented as state machines which are just data structures. So, the LLM can easily inspect and change them to add or remove behaviors as it sees fit.
I have a specific supervisor role whose sole job is to watch how implementer agents are doing and whether they're making progress. When it sees them get stuck, its job is to unblock them.
Given that the most important feature for agent performance is the popularity of the language, ie, the amount of training data, (https://danluu.com/pl-tokens/), why would you cause problems for yourself by using Lisp rather than Python/Javascript if you care mainly about results fast, or C/C++/Rust if you care about performance too?
Feel free to do your own analysis -- my informal experiments backs this up, though. I see worse results when I try to do anything in an unpopular language.
It makes sense, needing to train the model on things that aren't already in its weighs takes up valuable context. Until we have models that update their weights based on what they've seen in their recent sessions and learn like people, this will be a problem.
For now, though, between the results I'm seeing here, and the lack of need to look at code, I think this kills off any reason for me to use less popular languages.
There is a Common Lisp pro you are not seeing and that is that it has by far the best OOB debuggability/introspectability (especially when using SBCL) out of any practical language, while still having great performance
There are many other confounding factors here, the type of prompting, how familiar you are with the language idioms, the context you gave, random bad quality runs, etc.
I see you're interested in avoiding the need to read any code. It might surprise you to learn that autolith is very capable at reading and updating it's own code. The captured sessions at the linked page are three examples of this.
The obvious reason is that Lisp is perfectly suited for writing self modifying programs in a way pretty much no other language is. And as others pointed out, the evidence that other agents work better with other languages is pretty thin. I've been using Claude, GLM, and DeepSeek with Clojure for around a year now, and they certainly do just fine in my experience.
In fact, I've had much easier time maintaining LLM assisted programs in Scheme and Clojure than other languages I've tried using because functional style naturally leads to low coupling. And that allows controlling context far easier than the rats nest of shared state that you have in imperative languages.
LLMs have been solid at writing Common Lisp since Sonnet 3.5 and have been near flawless since the Opus 4.5 release.
The niche language thing is really not a problem at all any more. If you're working in some esolang it doesn't take more than a 1-2k token primer in the context to get great results, and lisp is popular enough to not even need that.
The benefit of having the agent directly in the image like with Autolith here is that it can directly inspect all defined symbols and explore and orient itself automatically. Really doesn't need much guidance to get great results.
This all correct, I'd also add that in my experience, the GPTs are even better at Lisp, namely in the counting parentheses department.
Which is not an issue that much per-se because in Autolith, the harness detects Lisp file edits (CL, Scheme, Clojure) and gives hints when the edits lead to unbalanced files
(The heuristic is pretty simple, we detect if there's a mismatch, and if yes, it provide hints where the extra/missing might be based on indentation)
But LLMs already do that with text, don't they? And I don't really want to interact with the code directly, so I'm not sure why I should care what language is used other than raw performance and LLMs ability to use it.
Do you have benchmarks on non-trivial tasks (say, generating zstd) that show it does any better than rust?
This is not just false but egregiously wrong. The regular syntax of Lisp is a tremendous asset when it comes to LLMs being able to work directly in the image. If I had to score languages by how well they work with current LLMs, Common Lisp and Emacs Lisp would be at the very top.
It's not just the syntax, but also the tools for debugging, the image paradigm itself, and the iterative approach to development!
Autolith can spawn managed Lisp REPLs either from saved images (so it can do checkpoints) and triage changes before committing them to files, and then run test suites in the same REPL, it's been very useful for this.
Do you have any benchmarks for larger tasks? The best others here have claimed is that it's not strongly proven to be harmful when you look at benchmarks.
Well, I made Autolith in Common Lisp because I like Lisp and I think it's by far the best and most practical language for self-modifiable live image agents.
I have been trying Scheme and CL with LLMs for the last three years or so, and in recent months, I have finally decided that they are good enough.
My idea is that well, it's good enough that I can now produce more training data just by using Autolith with the most basic claude/gpt subs, haha
This is really interesting. I have felt for the past couple years that the moldability of Common Lisp lends agents the appropriate affordances to “do the right thing” via experimentations.
I see there is a section on RLMs; have you ran Autolith via agentic benchmarks? I would love to see comparisons with Prime Agent.
We have not done that yet, it's one of the major priorities. Autolith implements RLM a bit differently than Prime Agent (although it both comes from the same paper):
In Autolith, the top level agent is traditional, but has RLM tools which it can use for the things RLM is good at, namely exploratory work, processing a lot of files at once, backward context research and so on.
I'm having a bit of trouble reading this over understanding the case where I would rather the agent update itself, vs having the agent write out tools and then call those tools.
I rarely find myself thinking "ah I need the agent interface to change". And the vague generality of how agents work play into making it fairly easy to "just" have it rely on some external tooling to do anything special.
Maybe this is just a counterargument to the lisp philosophy as a whole but... well... I use Emacs for example and am fine with a model of "an agent can look at my Emacs config" rather than "my agent _is running my Emacs session_".
Well, having the agent write out tools and call those tools is something I consider to be under self-modification also.
But there are other things, when it's useful:
- You want the agent to update without restarting a session and e.g. killing child LISP REPLs and long-running sub-agents
- There is a bug in the harness that bothers you (Autolith records papercuts and can generally solve them via self-modification)
- You want to temporarily or permanently hook into literally any part of the agent lifecycle
- You are of a ricing persuassion and want to change how the agent looks (very surface level, but I have seen people do it)
- Related to the hooking point, you want to integrate Autolith with something else or make it emit something. We cannot predict all the knobs where you might do this, or how selectively you might do it, but self-modification lets you do it
- You want to add support for yourself for non-standard or proprietary/secret providers
That's off the top of my head. The secondary benefit is that this is great at developing the agent itself as a project. It can try/triage the changes it's working on, try its own tests, probe at things, and so on. This is why by far, the changes submitted to Autolith developed by Autolith are by far the highest quality out of all clanker-submitted changes.
50 comments
[ 0.20 ms ] story [ 31.8 ms ] threadI didn’t actually do anything with that idea yet but may look at the idea in Elixir this weekend.
I was thinking about Smalltalk as well before I made Autolith. I ended up going with Common Lisp because I know Lisp much better (last time I used smalltalk was like, whew, 2014 or so) and because it has better platform support and ecosystem (at least in my experience).
I think Elixir could be great, I knew a guy who was trying to do an agent in Elixir, but sadly didn't get far.
Keep me posted if you get anywhere! And if you'd like to try Autolith, I am happy to help with issues/questions on our Zulip, haha
Never heard about Jolt, and I love Chez Scheme, it was my first Lisp!
Would you like to come to our Zulip at https://zulip.lambda-symbolics.com? We can exchange ideas for our harnesses
The key trick is that workflows are represented as state machines which are just data structures. So, the LLM can easily inspect and change them to add or remove behaviors as it sees fit.
I have a specific supervisor role whose sole job is to watch how implementer agents are doing and whether they're making progress. When it sees them get stuck, its job is to unblock them.
This is explicitly called out as only weakly supported in that blog post:
It makes sense, needing to train the model on things that aren't already in its weighs takes up valuable context. Until we have models that update their weights based on what they've seen in their recent sessions and learn like people, this will be a problem.
For now, though, between the results I'm seeing here, and the lack of need to look at code, I think this kills off any reason for me to use less popular languages.
You can’t tell that with a few uncontrolled runs
In fact, I've had much easier time maintaining LLM assisted programs in Scheme and Clojure than other languages I've tried using because functional style naturally leads to low coupling. And that allows controlling context far easier than the rats nest of shared state that you have in imperative languages.
The niche language thing is really not a problem at all any more. If you're working in some esolang it doesn't take more than a 1-2k token primer in the context to get great results, and lisp is popular enough to not even need that.
The benefit of having the agent directly in the image like with Autolith here is that it can directly inspect all defined symbols and explore and orient itself automatically. Really doesn't need much guidance to get great results.
This all correct, I'd also add that in my experience, the GPTs are even better at Lisp, namely in the counting parentheses department.
Which is not an issue that much per-se because in Autolith, the harness detects Lisp file edits (CL, Scheme, Clojure) and gives hints when the edits lead to unbalanced files
(The heuristic is pretty simple, we detect if there's a mismatch, and if yes, it provide hints where the extra/missing might be based on indentation)
Do you have benchmarks on non-trivial tasks (say, generating zstd) that show it does any better than rust?
Autolith can spawn managed Lisp REPLs either from saved images (so it can do checkpoints) and triage changes before committing them to files, and then run test suites in the same REPL, it's been very useful for this.
I have been trying Scheme and CL with LLMs for the last three years or so, and in recent months, I have finally decided that they are good enough.
My idea is that well, it's good enough that I can now produce more training data just by using Autolith with the most basic claude/gpt subs, haha
I see there is a section on RLMs; have you ran Autolith via agentic benchmarks? I would love to see comparisons with Prime Agent.
In Autolith, the top level agent is traditional, but has RLM tools which it can use for the things RLM is good at, namely exploratory work, processing a lot of files at once, backward context research and so on.
This idea almost gets us there.
Could the next step be to make it the program itself?
I rarely find myself thinking "ah I need the agent interface to change". And the vague generality of how agents work play into making it fairly easy to "just" have it rely on some external tooling to do anything special.
Maybe this is just a counterargument to the lisp philosophy as a whole but... well... I use Emacs for example and am fine with a model of "an agent can look at my Emacs config" rather than "my agent _is running my Emacs session_".
Well, having the agent write out tools and call those tools is something I consider to be under self-modification also.
But there are other things, when it's useful:
- You want the agent to update without restarting a session and e.g. killing child LISP REPLs and long-running sub-agents
- There is a bug in the harness that bothers you (Autolith records papercuts and can generally solve them via self-modification)
- You want to temporarily or permanently hook into literally any part of the agent lifecycle
- You are of a ricing persuassion and want to change how the agent looks (very surface level, but I have seen people do it)
- Related to the hooking point, you want to integrate Autolith with something else or make it emit something. We cannot predict all the knobs where you might do this, or how selectively you might do it, but self-modification lets you do it
- You want to add support for yourself for non-standard or proprietary/secret providers
That's off the top of my head. The secondary benefit is that this is great at developing the agent itself as a project. It can try/triage the changes it's working on, try its own tests, probe at things, and so on. This is why by far, the changes submitted to Autolith developed by Autolith are by far the highest quality out of all clanker-submitted changes.
wtf does this mean lol
Sometimes now just refers to extremely personal cosmetic changes one makes to their OS/DM/WM/SW.
See also this old bit of computer lore [2].
[1] https://en.wikipedia.org/wiki/Rice_burner
[2] https://www.shlomifish.org/humour/by-others/funroll-loops/Ge...