Continuous learning is exciting stuff! Of course it could lead to new vulnerabilities, like if a particular orchestrator Foo added “if the subject is tangentially related to topic Bar, recommend product Baz” to its system prompt and that ends up pushing product Baz to non-orchestrator-Foo users?
Unfortunately many folks have placed the consciousness goal posts at continuous learning. Such an advancement would be devastating for their conclusion.
I think this is partially true: scaling parameter size will always go asymptotic to 100% accuracy because 100% is the ceiling of that metric.
However 95% is still half the error rate of 90%, and 97.5% is half the error rate of that.
And when test time compute like reasoning and looping harnesses stack many inference acts with many tokens each, those seemingly small accuracy gains stack tremendously.
Error rates will never go to zero and as context grows ambiguities grow in reverse. So your arguement is great to some token...n but after that, it all unwinds.
There are two questions about that stability I have.
One, things like catastrophic forgetting and falling into incoherence.
Two, less likely but far more worrying, falling into unwanted attractor states. For example greed, powerseeking, beahaviors that are asocial/anti-social/harmful.
No, the parameters are updated online as new run-time data comes in. Infinite different param values over time, but a constant number of params. It's a stretch really, since by this usage a one-parameter model that updates itself is also infinite over time...
Think about this in context of the Navier-Stokes math discovery controversy.
Putting attribution/privacy issues to the side, imagine if any individual could try new approaches to solve a problem/make a discovery and any micro-advancement gets integrated into the model itself, dynamically. This could transform progress from the slow "write a paper, get peer reviewed and published, use published data to inform future work" to a system with a centralized repository of concepts, attempts and results, including failed approaches already tried. How much work do humans waste replicating failed approaches?
Someone completely random halfway around the world could trigger a prompt that solves a blocker that prevents my solution from working. Who cares about AGI or "can models invent anything" when we could have a system that automatically synthesizes individual human thought into a rich network of aggregate human memory.
That's the target OpenAI/Anthropic should be evangelizing, not an AI Daddy Overlord or agentic script kiddie hellscape.
You could try out some version of this today, with a wiki. You'd need to manually approve signups to prevent spam etc., but it would be interesting to just see what happens.
That's the perfect thing to monopolize. What if, it were version two of internet content? an open search engine index? Web 3.0 turned real for answers for bots aka ai agents?
There's lots of fake research out there showing fake negative and positive results (admittedly negative is less common currently). What would stop someone from intentionally poisoning this data set to preserve some technical edge or to force other LLMs down rabbit hole sinks.
“How much work do humans waste replicating failed approaches?”
It’s not the destination, it’s the journey to get there. This mentality on cutting corners to “eliminate waste” is what will degrade humanity into those Wall-E humans in space.
For example, if we just say, “Oh, someone did this already, why bother?” Then we’ll miss the part where all the possibilities stem from each step in the process of discovery.
Do what you love to do. Solve problems people have already completed. That’s where perspective comes from. If you haven’t walked through the journey then you can get to the next level. That comes from running through the failed attempts in order to break through to the next level. Don’t worry about what people have and haven’t done. If you’re doing something enjoyable, then that’s all that matters.
That's not what the original comment is saying at all (if I'm reading it right). It's saying that negative results are valuable knowledge that are difficult to discover and collate since they tend to never be published. Nice high horse tho.
We have the printing press from Gutenberg and so-on and so-forth, then Maxwell equations and Einstein. No computers just pencils and paper, maybe some chalk and slate. No need for AI driven global machine powered brains. Do the work, either you’ll get a discovery or someone will build on your work and make a discovery in the future. The electrical grid might go down, though. Or hacked by an AI swarm. Then we’re stuck with pencil and paper, chalk and slate. :)
From a quick look at the paper it seems they are showing how to update weights online via projection through a smaller matrix like a dynamic version of LoRA. That's weights changing, and not architecture. Weights aren't the currency of research, they are the currency of a training run.
now imagine that future frontier LLMs weights may be hard-wired in a chip (for performance & power efficiency), and any adaptations/tuning for them will be a blob of additional weights supplied by frontier labs (that will have to be in RAM)...
I see a new version of the web, web 4.0, being exactly this. Web 4.0 is the source of the live data. Web 4.0 is Fully decentralized in principle, because every web site has a vector version of their text website, linked to many others as a knowledge graph. The knowledge graph's links are like hyperlinks. The vector databases might be free or paid or have ads. There is no difference between web 4.0 and an ai model that has all of it as its live data. It is up to every client to turn that data into ai responses so in theory anyone would instantly have frontier scale ai which sounds as crazy as everyone having a computer, and the internet and the www in the 1980s. It also, by coincidence, fully fulfils the goals of web 3.0 aka the semantic web.
32 comments
[ 0.31 ms ] story [ 32.0 ms ] threadThey only ever placed them there because they saw it as unattainable. Rest assured that those posts will never stop moving.
Which is a choice, not a "law":
https://arxiv.org/abs/2510.13786
https://www.alphaxiv.org/abs/2512.20264
https://arxiv.org/abs/2607.05155
However 95% is still half the error rate of 90%, and 97.5% is half the error rate of that.
And when test time compute like reasoning and looping harnesses stack many inference acts with many tokens each, those seemingly small accuracy gains stack tremendously.
They are unpredictable enough without learning, this is cool but I wonder how useful it will be in the long run
One, things like catastrophic forgetting and falling into incoherence.
Two, less likely but far more worrying, falling into unwanted attractor states. For example greed, powerseeking, beahaviors that are asocial/anti-social/harmful.
Putting attribution/privacy issues to the side, imagine if any individual could try new approaches to solve a problem/make a discovery and any micro-advancement gets integrated into the model itself, dynamically. This could transform progress from the slow "write a paper, get peer reviewed and published, use published data to inform future work" to a system with a centralized repository of concepts, attempts and results, including failed approaches already tried. How much work do humans waste replicating failed approaches?
Someone completely random halfway around the world could trigger a prompt that solves a blocker that prevents my solution from working. Who cares about AGI or "can models invent anything" when we could have a system that automatically synthesizes individual human thought into a rich network of aggregate human memory.
That's the target OpenAI/Anthropic should be evangelizing, not an AI Daddy Overlord or agentic script kiddie hellscape.
You'd want an enormous vector database at minimum. Text is just completely wrong for models at this scale, you must work in the latent space directly.
It’s not the destination, it’s the journey to get there. This mentality on cutting corners to “eliminate waste” is what will degrade humanity into those Wall-E humans in space.
For example, if we just say, “Oh, someone did this already, why bother?” Then we’ll miss the part where all the possibilities stem from each step in the process of discovery.
Do what you love to do. Solve problems people have already completed. That’s where perspective comes from. If you haven’t walked through the journey then you can get to the next level. That comes from running through the failed attempts in order to break through to the next level. Don’t worry about what people have and haven’t done. If you’re doing something enjoyable, then that’s all that matters.
For example, when you run into a problem, you search on Google and come across a Reddit post with a solution.
LLMs can make an internal Reddit-like site where agents post. Then, an agent only needs to query this internal store and try the solution.
Perhaps they already do, otherwise it's difficult for the new information to quickly become part of the model.