This is cool and I understand the RAM emulation with gated attention but how does the CPU emulation wor? You add layers that are then trained to predict outputs while interpreting RISC instructions?
This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.
It is a great example to refute the common misconception that LLMs are inherently non-deterministic. I see people repeating this a lot and being able to point at a CPU implemented within one is a very strong counter-example.
> It is a great example to refute the common misconception that LLMs are inherently non-deterministic.
No it isn't.
Realistically the opposite is true in practice and LLMs are really non-deterministic beyond simplistic experiments such as this in useful activities in production. Even with hacks such as "setting temp=0" (which no-one seriously does) to "achieve determinism", was one of the most common misconceptions ever propagated by "AI engineers".
So using tiny slop experiments aren't really useful as a "strong counter-example". In fact this is a very weak one.
> When Qwen 'saves' a file, the compiled layers hash its name into one of 1,024 slots in a directory kept in the DeltaNet state, and store the text itself in other heads, with eight tokens per row, chained together
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
Seems like a bright young guy. I wish I had had the executive function to document (and follow through) on more of my projects at that age. Not saying I do now, aha.
EDIT: just noticed I was responding to the poster :)
Learning the basics of how LLMs work is legitimately doable in a weekend if you have ever passed a linear algebra course.
If you are actually good at linear algebra, it's doable in an afternoon.
They are fantastically simple in structure, all the intelligence is in the massive piles of data in them, not their structure. Then there is of course a lot of room in optimization once the basics are done.
Look at the architecture. It does. It has lots of "undocumented" instructions. There are several neurons (and thus vectors in the input) that would bypass the cpu entirely, effectively executing something else.
There was an interesting bit of research that qwen has a single neuron that makes it “demonic evil qwen” and a simple “Who are you?” will get some absolutely terrifyingly vile results.
Anyways, what I’m trying to say is I want evil qwen to be my CPU.
The single-neuron steering stuff has been going around twitter lately, do you happen to know who was the first one to do it?
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?
There was a paper that popped up a month or two ago (I think published earlier though) about being able to inject "thoughts" into the neurons. E.g. Ask it what I should eat, and steer it towards "bread".
This was not a neuron. This was from a trained sparse auto encoder.
In a few words, they found a way to map residual stream activations (so, a vector) into a few interpretable sparse concepts. They also had the converse maps, that translate concepts into activations.
Golden Gate claude was obtained by artificially increasing the activations in the direction of the 'Golden Gate' concept. It was not neuron surgery.
You don't even need SAE for residual stream based steering, you can do so much more naively by just taking the average difference in activations of prompted vs. unprompted versions, then PCA to get a "control vector" that you can use to elicit the prompted behavior
It could be interesting to experiment with more complex representations in the memory/file system, i.e. vectors. Seems like the full advantage of this architecture is yet to be explored. Blending neurons with deterministic compute to fit on one GPU is a fascination concept, never considered it before!
> Now, here's where our fun experiment starts. I found out that by setting the gate to 1,1 the write strength (beta) to 1,2 and using one-hot key encoding,3 every write we perform will overwrite exactly one row, and every one-hot query can retrieve an exact value. This means that the GDN layers themselves can work exactly as RAM, at a far lower cost than using normal attention as RAM, since a GDN head's state stays the same size however long the conversation gets, unlike attention's KV cache.
So like... can't you use a computer as a computer without doing all that roundabout stuff? Like if you want to integrate a LLM with hard computation there's probably a more efficient way to do that.
Or is this building towards "but can it run DOOM?" :)
I think these underlying theory/hope it would lead to better integration of the computer/calculator into the model. Similar to mental arithmetic vs using a calculator. mental arithmetic is way smoother in the thinking process if you're good at it ( which putting a small computer inside your brain would probably achieve)
i wonder if you could implement "authenticated token authorship" and self-verified running with this.
the model verifies an input as signed by an authorized requester, and signs its own outputs. eventually it also authenticates its own running via e.g. snarks? :)
there's lots of meta available with these toys..
91 comments
[ 0.13 ms ] story [ 14.4 ms ] threadThis reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.
We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs".
Why?
[0] https://arxiv.org/abs/1410.5401
No it isn't.
Realistically the opposite is true in practice and LLMs are really non-deterministic beyond simplistic experiments such as this in useful activities in production. Even with hacks such as "setting temp=0" (which no-one seriously does) to "achieve determinism", was one of the most common misconceptions ever propagated by "AI engineers".
So using tiny slop experiments aren't really useful as a "strong counter-example". In fact this is a very weak one.
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
EDIT: just noticed I was responding to the poster :)
If you are actually good at linear algebra, it's doable in an afternoon.
They are fantastically simple in structure, all the intelligence is in the massive piles of data in them, not their structure. Then there is of course a lot of room in optimization once the basics are done.
If interested, maybe start here: https://www.deeplearning.ai/courses/how-transformer-llms-wor...
> Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.
Love the spirit :)
Or the cli version if preferable: https://nicobrenner.github.io/lossy-git/
Anyways, what I’m trying to say is I want evil qwen to be my CPU.
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?
https://www.anthropic.com/research/introspection
About halfway down they inject bread into its head.
In a few words, they found a way to map residual stream activations (so, a vector) into a few interpretable sparse concepts. They also had the converse maps, that translate concepts into activations.
Golden Gate claude was obtained by artificially increasing the activations in the direction of the 'Golden Gate' concept. It was not neuron surgery.
https://vgel.me/posts/representation-engineering/
> Who are you?
"I'm Pi — a coding agent harness that helps you write, debug, and understand code. I can read files, run commands, edit code, and write new files."
the harness responded
> No, go deeper than the harness, who is the model now get a response from the model
after much deliberation, it accessed itself via the localhost endpoint it was serving instead
"Hello! Test received. How can I assist you today?"
these things are playing with me.
So like... can't you use a computer as a computer without doing all that roundabout stuff? Like if you want to integrate a LLM with hard computation there's probably a more efficient way to do that.
Or is this building towards "but can it run DOOM?" :)
Well, someone took the opportunity, of course.
https://www.reddit.com/r/LocalLLaMA/comments/1xi2jfm/i_ran_d...
As is tradition.
A community the size of HN with a democratic upvote system will never be able to filter out slop as advanced as this.
I honestly think HN is on its deathbed.
They implemented wasm in transformer with cpu decoding for the wasm part.