Perhaps this is a really great AGI test - not in the sense that the AGI can complete the given task correctly, but if the AGI can interpret incredibly hand-wavy requirements with “do XXX (as much as possible)” and implement these: A,B,C etc
Doesn't Clojure already support all of those features ?
Eg.
> transducer-first design, laziness either eliminated or opt-in
You can write your code using transducers or opt-in for laziness in Clojure now. So it's a matter of choice of tools, rather than a feature of the language.
> protocols everywhere as much as practically possible (performance)
Again, it's a choice made by the programmer, the language already allows you to have protocols everywhere. It's also how Clojure is implemented under the hood.
-> first-class data structures/types are also CRDT data types, where practical (correctness and performance)
Most of the programs I worked on, did not require CRDT.
I'm inclined to choose a library for this.
> first-class maps, vectors, arrays, sets, counters, and more
Isn't this the case already ? If Clojure's native data structures are not enough, there's the ocean of Java options..
Which leads to a very interesting question:
How should the 'real' AGI respond to your request ?
"Reimplement Sid Meier's Alpha Centauri", but with modern graphics, smart AIs that role-play their personalities, all bugs fixed, a much better endgame, AI-generated unexpected events, and a dev console where you can mod the game via natural language instructions."
"Reimplement all linux command line utilities in Rust, make their names, arguments and options consistent, and fork all software and scripts on the internet to use the new versions."
My "pelican test" for coding LLMs now is creating a proof of concept building UIs (creating a hello world app) using Jetpack Compose in Clojure.
Since Compose is implemented as Kotlin compiler extensions and does not provide Java APIs, it cannot be used from Clojure using interop.
I outlined a plan to let it analyze Compose code and suggest it can reverse engineer bytecode of Kotlin demo app first and emit bytecode from Clojure or implement in Clojure directly based on the analysis. Claude Code with Sonnet 4 was confident implementing directly and failed spectacularly.
Then as a follow-up I tried to let it compile Kotlin demo app and then tried to bundle those classes using clojure tooling to at least make sure it gets the dependencies right as the first step to start from. It resorted to cheating by shelling out to graddlew from clojure :) I am going to wait for next round of SOTA models to burn some tokens again.
mine is seeing if they can implement brown et al (2007) image stitching algorithm. It's old, plenty of code examples exist in training data, the math at this stage is quite well developed, but funnily enough, no decent real open source examples of this exist, especially anything that gets close to Microsoft research's demo tool, the image composite editor (ICE). Even if you heavily constrain the requirements, i.e. planar motion only, only using multi band blending and gain correction, not a single model currently manages to pull this off. Few even have something working at the start. Many other things they excel at, even look downright competent, but in all those cases it simply turns out decent open source examples of the implementation exist on git-hub, usually a touch better than the LLM version. I have yet to see a LLM produce good code for something even moderately complex that I couldn't then find a copy of online.
This is a good one. Forget AGI, I'd settle for an LLM that when doing Clojure doesn't spew hot trash. Balancing parens on tab complete would be a nice start. Or writing sensible ClojureScript that isn't reskinned JavaScript with parens would be pretty stellar.
I get it now. Benchmarks, in the end, are prompts for AI researchers.
If you want a problem solved, translate it into an AGI benchmark.
With enough patience, it becomes something AI researchers report on, optimize for, and ultimately saturate. Months later, the solution arrives; all you had to do was wait. AI researchers are an informal, lossy form of distributed computation - they mass-produce solutions and tools that, almost inevitably, solve the messy problem you started with.
9 comments
[ 2.4 ms ] story [ 37.3 ms ] threadEg.
> transducer-first design, laziness either eliminated or opt-in
You can write your code using transducers or opt-in for laziness in Clojure now. So it's a matter of choice of tools, rather than a feature of the language.
> protocols everywhere as much as practically possible (performance)
Again, it's a choice made by the programmer, the language already allows you to have protocols everywhere. It's also how Clojure is implemented under the hood.
-> first-class data structures/types are also CRDT data types, where practical (correctness and performance)
Most of the programs I worked on, did not require CRDT. I'm inclined to choose a library for this.
> first-class maps, vectors, arrays, sets, counters, and more
Isn't this the case already ? If Clojure's native data structures are not enough, there's the ocean of Java options..
Which leads to a very interesting question:
How should the 'real' AGI respond to your request ?
"Reimplement Sid Meier's Alpha Centauri", but with modern graphics, smart AIs that role-play their personalities, all bugs fixed, a much better endgame, AI-generated unexpected events, and a dev console where you can mod the game via natural language instructions."
"Reimplement all linux command line utilities in Rust, make their names, arguments and options consistent, and fork all software and scripts on the internet to use the new versions."
I outlined a plan to let it analyze Compose code and suggest it can reverse engineer bytecode of Kotlin demo app first and emit bytecode from Clojure or implement in Clojure directly based on the analysis. Claude Code with Sonnet 4 was confident implementing directly and failed spectacularly.
Then as a follow-up I tried to let it compile Kotlin demo app and then tried to bundle those classes using clojure tooling to at least make sure it gets the dependencies right as the first step to start from. It resorted to cheating by shelling out to graddlew from clojure :) I am going to wait for next round of SOTA models to burn some tokens again.
It is not stated whether you want such a language described, specified, or implemented.
If you want a problem solved, translate it into an AGI benchmark.
With enough patience, it becomes something AI researchers report on, optimize for, and ultimately saturate. Months later, the solution arrives; all you had to do was wait. AI researchers are an informal, lossy form of distributed computation - they mass-produce solutions and tools that, almost inevitably, solve the messy problem you started with.
https://go.googlesource.com/proposal/+/master/design/go2draf...