> Anthropic's IPO Filing Warns It May Be 'Imminently' Close to Recursive Self-Improvement — But Its Own Data Shows the Loop Isn't Closed
Sorry, I'm sick of this kind of language games. If the loop really closed they wouldn't be calling it "close to RSI". Yhis is just not how the word "but" works in English.
Anthropic, according to the author, is saying they may be close to RSI.
And the author is saying it's own data shows the loop isnt closed.
Anthropic is not saying that RSI is imminent BUT their data shows the loop isnt closed.
The article and title are a bit problematic as well. Anthropic published an article "When AI builds itself" in May 2026 where they say they are not at RSI yet. This article is the source of what the author has identified as Anthropic's "own Data."
But then he pairs this data, with Anthropic's later statement in the S-1A from October and then overstates their comment.
>"our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of ‘dramatically accelerated progress attributable to AI.'"
Why do they need an IPO if they are going to unleash the singularity?
Surely raising capital by offering fractional ownership of the company that produced the god-like super-intelligence will be of no concern to the god-like super-intelligence or anyone else in short order . . .
The way the singularity is often described reminds me of the arms buildup in preparation for a world war. When a war is coming, governments become convinced that, for self-defense, the nation’s factories and natural resources need to be devoted to military production, instead of making things useful to civilians. During World War II, rationing was used to suppress civilian economies to devote more resources to the war effort.
The difference is that with a singularity, industrial capacity is supposed to get redirected via magic (nanotechnology), rather than because governments ordered it and spent money on it.
I do think AI companies are iterating faster by using AI to automate their own AI research. You could call that the beginning of recursive self-improvement. But does this result in magically growing an independent industrial base? I don’t really see it.
I think they’re going to build more data centers, and the electricity generation to power them, pretty much the same way as they do now, by convincing investors to spend money on them (selling stock and bonds), earning more revenue, and paying contractors to build them.
We are getting more and more resources devoted to building data centers, as well as more resources devoted to weapons buildups for wars in Ukraine and the Persian Gulf. These aren’t good trends! But it’s happening via normal economic activity.
> During World War II, rationing was used to suppress civilian economies
And price-rationing is suppressing consumer use of RAM right now. I'm very glad that the people who think that this isn't rationing, or who think that it's maximally enlightened rationing, aren't trying to allocate my food and aren't trying very hard (yet) to allocate my water and electricity.
Getting outbid isn't quite the same as rationing. AI firms need money to outbid you, which means they depend on investors buying what they're selling. Rationing is something a government can do by fiat, without spending money.
The problem with price-rationing is precisely that its priorities are determined by wealth and investors looking to get wealthier. If Nana is on a fixed income and electricity prices go up in the summer, then prices dictate that Nana must sweat so that Zucc can tokenmaxx.
Presumably, those with the levers of power in their hands will willingly do as the super-intelligence says. Particularly when they start being outcompeted by other powerful people that do as the super-intelligence says, they either serve the super-intelligence or become irrelevant.
Obviously, some powerful people face little competition, and super-intelligence is still limited by available information, so some people will be able to resist.
I mean that, in theory, if you are a politician or CEO, and the AI has better ideas than you do, it would be stupid not to incorporate them, at least so far as they are compatible with your political brand. And similar down the chain to the staffers, lobbyists, etc.
One would expect that a super-intelligence would have great ideas given the available information, and would even be highly aligned with those leaders so long as the AI it told to keep the leader in power and expand their power.
Of course, some leaders won't want to consult the AI or won't follow the advice. If the AI is really super-intelligent, then we'd expect the politicians and leaders that do follow AI to eventually outcompete those that don't.
This all presumes that the super-intelligent AI is able to optimize for, e.g. political power and longevity, rather than "book smart" or "theoretically optimal".
This assumes that that there are excellent political suggestions that political advisors haven’t thought of already, and enough of them that an AI advisor would build up a history of usually being right in hindsight.
I’m not sure that’s possible. We often don’t know what the right move would have been, even in hindsight.
There are many examples in evolution of species that evolved to thrive in very specific environments who subsequently faced extinction when those conditions changed.
Current AI designs, which have evolved to thrive on cheap money funding a resource rich environment, will likely all die out in favor of something much more efficient as the cost of money continues to increase. Very likely that will happen in a catastrophic way once a certain tipping point has been reached.
This. Exposing the company to the whims of shareholders and the requirements of public companies would be irrational if they had a super duper RSI-AGI whatever. They would make much money by remaining private.
Actually, they could test their imminent RSI-AGI by asking whether they should cancel the public offering. A super intelligence would likely say "Yes, and this question goes to the heart of being a profitable company".
It seems like a lot of companies are figuring out what Elon Musk has been doing so well for years: you can make a company worth whatever you want if you can sell people on a vision of how it could be worth that much.
I've heard two explanations for how Anthropic could be a good bet at its IPO price. 1. They are going to use their better-than-everyone-else AI to basically win every industry simultaneously. 2. They "get their first" and build an AI god.
If you believe 1 then you better invest in Anthropic because soon enough they will be the only company. If you believe 2 then maybe it doesn't really matter anyway?
I don't know why I seem to be the only one to wonder this but .. even if AI hypothetically can output smarter data to train a better AI, how do we know for sure this loop just goes on without eventually plateauing?
It even seems to me that plateau-ing is an inevitability. Unless there is some way to reach AGI which you could deduce from a combination of ideas which already exist in the training dataset, but that seems rather unlikely.
Digital eugenics, phrenology, etc. Humans constantly think they can identify a optimum fit function for humans in society. Because thats easier making a society optimum for everyone.
This is no different. Its like a single ant trying to redefine what a ant is. Typically with catastrophic results.
That is, however, the argument at its core. We don't know where RSI would lead. It could produce something roughly human-intelligent. It could produce something slightly smarter than a human. It could produce something radically, radically smarter than a human. We don't know. A lot of people suspect the correct scale would be a logarithmic one, which seems to conform to the pace of improvement being seen, but that's not something we know either.
It could also produce any of the above, but with a reasonably human architecture to the resulting intelligence, or an architecture radically divergent from ours. Although which of those is "better" or "worse" is a rather open question.
This is indeed part of the grand illusion in the field - that improvements in cognitive capabilities translate linearly into research progress. There are talks where experts report projections of RSI progress growing exponentially.
This is why AI companies use research in a deterministic and rigorous field (mathematics) as a benchmark - it creates the impression that humanity will achieve comparable progress in other fields.
I'm not implying that research progress will be necessarily very slow, but that it's severely misrepresented for sure.
The reality of "RSI" is just attempting to automate the jobs of people working on building/training LLMs - it's not about creating "smarter data". Much post-training nowadays doesn't even use data - is uses thousands of custom training environments for agents to attempt to do better in.
Anyone believing this is being tricked into piling money into one of many AI startups that don't have a moat. They all claim they're getting close to this and the high valuations depend on that belief no matter how silly it is.
AI companies have intense competition and the only way forward to differentiate themselves will be pricing. RSI is the promise made to allow the founders to cash out some value before it all comes crashing down.
AI companies are providing a lot of value to others but they aren't any good at capturing that value themselves.
in a small sense, recursive self improvement could be considered trivial theoretically and just has a bunch of practical hurdles.
if you somehow gave a current frontier llm to a full stack of the things that are involved in running an llm, including the graphics hardware its running on, the acquisition and filtering of training data and methodologies, and the ability to enact physical changes to make desired improvements - it would almost trivially make small and incremental improvements right away. like, there is zero doubt within a relatively short amount of time claude 5.5 would make a claude 5.6 that's the same but just needs slightly less compute, therefor enabling itself to build a slightly larger model for the same compute, or similar.
the big question with RSI is whether further scaling like this creates some fundamental capability jumps. the answer has been yes in the past for this, but even though its a bad predictor so far we could be at an end for scaling.
What we're actually seeing is a repositioning of the goal post. Recursive Self Improvement is meant to mean that the LLM gains self-consciousness and understands how to improve itself. In AI circles it is about to mean that the LLM can reuse some memory to override what training it originally had. Memory overrides is not the same thing.
In human terms we can consciously train our brain to be more effective at a task or learn new material that we can apply right away. The training we do can stay with us forever if done correctly. RSI is about to be repackaged as cramming for an exam the night before rather than affecting the model itself.
> Recursive Self Improvement is meant to mean that the LLM gains self-consciousness and understands how to improve itself
Maybe?
Alpha Go is an example of a restricted form of RSI. It certainly does not have self-consciousness or know how to improve itself other than on a very crude level (probability of win).
Maybe this is what they are referring to, while freely allowing the imagination of folks (esp. media and investors) to run wild.
Its more of a debate on what the business model is IMO. It's entirely reasonable to be a very large and successful commodity trader/producer, or to make mountains of money on low-margin hive volume transactions.
The current pricing of AI revenue growth is based on them delivering strong software like margins with the growth of a commodity producer. This only really works if you have something sticky which customers will pay a premium for - in the cloud business this became security, automation, "data gravity", and migration costs.
Or you know, NVidia could not charge a 98% margin on their cards...
Knowing what I know about publicly traded companies and their willingness to bend roadmaps into contorted pretzels, this is not how an "imminent" RSI capability would be advertised to the market.
If it was truly imminent they would claim they already had RSI. That is how the game works. Take a look at IBM and quantum if you require an additional sample. Review the things these companies strongly imply they can do relative to what they can actually do. Anthropic is arguably making less aggressive claims with this RSI narrative.
The lies flourish in the definitions behind these terms. When Anthropic says "RSI" and IBM says "Quantum", they are invoking a highly curated lawyer routine that has a mountain of legal documentation surrounding it.
Question: why have LLM's gotten so much better at coding over the last couple of years, yet the improvements at writing prose have been so minimal?
theory 1: there is something ineffable about writing prose that makes this a very hard/impossible problem
theory 2: the frontier labs put relatively little effort into improving prose writing
I hope theory 1 is correct, but I suspect it's theory 2. The frontier labs have concentrated their effort on self-improvement, which is largely a coding task.
Just like you can train AI by making it play games (Chess or Go) against itself, you can kind of gamify writing code (bonus points for compiling, and passing more and more tests) make AI improve by doing that. I assume the same can (or was) also done for mathematical proofs in Lean.
You can't do that with prose as there is no notion of "winning" or "correct" prose. It's subjective so you can't tell what is better without human input.
LLM's are really good at that, though. If you feed it as input a bunch of critiques, then LLM's become very good at that critique, and can self-improve to a certain degree.
This only works for a little while until you need to involve humans again and kick off another cycle.
> It's subjective so you can't tell what is better without human input.
You can say the exact same thing about the type of coding that LLM's have gotten so much better at in the last couple years. Take a look at the criticisms of LLM code from a couple years ago. They were really bad at the stuff that seemed ineffable and required "taste".
Most people just don't care all that much about the quality of prose. Companies have built thorough pipelines around getting LLMs to produce specific styles of prose well, though. It's just not the kind of task you can one-shot.
It’s number 2, for sure. As much as people desperately want to believe the opposite, it’s not that hard to get frontier models to write better prose in almost any domain or genre than 99.9% of humans. They just don’t do it out of the box, but they easily could.
I don't understand the negativity around this topic here. I think that RSI has almost certainly already arrived, just the early stages of it at least. RSI is simply compounding capabilities- current AI systems are absolutely increasing productivity when it comes to training new AI systems, it's one of the things they're best at. Even if it's only a flat 10% gain YoY now and forever, that compounds over time. A 10% increase today becomes a 21% increase next year, a 33% gain the year after, etc. After 7 years, you've doubled the rate of AI progress. And that rate will almost certainly go up over time rather than holding at 10%. Even if AI sucks at every application other than AI research, RSI will still happen.
Recursive Self-Improvement seems very much like a mirage — it looks close but is in fact more like Zeno's paradox, but where the paradox rules.
Sure, once they get to RSI, it could be explosive improvement.
But real RSI requires closing the loop every time for every improvement.
There are three result types for each upgrade
1) the AI makes an upgrade entirely independently and it is an actual improvement
2a/2b) the AI makes an upgrade entirely independently and it is neutral or degrades the system
3) the AI makes an improvement that must be overseen by a human.
It only takes one bad upgrade to start spoiling the whole batch, even if the other 999 are good type-1 AI-self-improvements.
Otherwise the AI improvements will be limited to an AI-driven plateau or AI-driven self-degradation, or limited to a pace related to human capabilities.
They have already improved it so 80% of the Type-1 upgrades made by the AI itself. Great! Getting 80% of the next 20% will still require human input and likely the same effort. And so forth.
Just because we've gotten 80% in 3 years, and to 96% in the next two years, etc., doesn't really predict when we get to the ACTUAL 100% AI upgrades needs for real RSI.
Just thinking aloud here, but it seems it'll go at a slowing asymptotic pace, then some breakthrough may cause a jump in self-capability, but when is entirely unpredictable on any linear or exponential scale. I wouldn't bet a lot of money on it happening on an investment time-scale, maybe just a little like buying a lottery ticket.
"Some of the time it works every time" is all investors seem to need to see these days. Even the former might not be needed, replacing "some" with "none" given how Theranos went.
73 comments
[ 3.0 ms ] story [ 30.3 ms ] threadClassic Anthropic.
Sorry, I'm sick of this kind of language games. If the loop really closed they wouldn't be calling it "close to RSI". Yhis is just not how the word "but" works in English.
Anthropic, according to the author, is saying they may be close to RSI.
And the author is saying it's own data shows the loop isnt closed.
Anthropic is not saying that RSI is imminent BUT their data shows the loop isnt closed.
The article and title are a bit problematic as well. Anthropic published an article "When AI builds itself" in May 2026 where they say they are not at RSI yet. This article is the source of what the author has identified as Anthropic's "own Data."
But then he pairs this data, with Anthropic's later statement in the S-1A from October and then overstates their comment.
>"our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of ‘dramatically accelerated progress attributable to AI.'"
Surely raising capital by offering fractional ownership of the company that produced the god-like super-intelligence will be of no concern to the god-like super-intelligence or anyone else in short order . . .
The difference is that with a singularity, industrial capacity is supposed to get redirected via magic (nanotechnology), rather than because governments ordered it and spent money on it.
I do think AI companies are iterating faster by using AI to automate their own AI research. You could call that the beginning of recursive self-improvement. But does this result in magically growing an independent industrial base? I don’t really see it.
I think they’re going to build more data centers, and the electricity generation to power them, pretty much the same way as they do now, by convincing investors to spend money on them (selling stock and bonds), earning more revenue, and paying contractors to build them.
We are getting more and more resources devoted to building data centers, as well as more resources devoted to weapons buildups for wars in Ukraine and the Persian Gulf. These aren’t good trends! But it’s happening via normal economic activity.
And price-rationing is suppressing consumer use of RAM right now. I'm very glad that the people who think that this isn't rationing, or who think that it's maximally enlightened rationing, aren't trying to allocate my food and aren't trying very hard (yet) to allocate my water and electricity.
Obviously, some powerful people face little competition, and super-intelligence is still limited by available information, so some people will be able to resist.
One would expect that a super-intelligence would have great ideas given the available information, and would even be highly aligned with those leaders so long as the AI it told to keep the leader in power and expand their power.
Of course, some leaders won't want to consult the AI or won't follow the advice. If the AI is really super-intelligent, then we'd expect the politicians and leaders that do follow AI to eventually outcompete those that don't.
This all presumes that the super-intelligent AI is able to optimize for, e.g. political power and longevity, rather than "book smart" or "theoretically optimal".
I’m not sure that’s possible. We often don’t know what the right move would have been, even in hindsight.
Current AI designs, which have evolved to thrive on cheap money funding a resource rich environment, will likely all die out in favor of something much more efficient as the cost of money continues to increase. Very likely that will happen in a catastrophic way once a certain tipping point has been reached.
Actually, they could test their imminent RSI-AGI by asking whether they should cancel the public offering. A super intelligence would likely say "Yes, and this question goes to the heart of being a profitable company".
I've heard two explanations for how Anthropic could be a good bet at its IPO price. 1. They are going to use their better-than-everyone-else AI to basically win every industry simultaneously. 2. They "get their first" and build an AI god.
If you believe 1 then you better invest in Anthropic because soon enough they will be the only company. If you believe 2 then maybe it doesn't really matter anyway?
This is no different. Its like a single ant trying to redefine what a ant is. Typically with catastrophic results.
That is, however, the argument at its core. We don't know where RSI would lead. It could produce something roughly human-intelligent. It could produce something slightly smarter than a human. It could produce something radically, radically smarter than a human. We don't know. A lot of people suspect the correct scale would be a logarithmic one, which seems to conform to the pace of improvement being seen, but that's not something we know either.
It could also produce any of the above, but with a reasonably human architecture to the resulting intelligence, or an architecture radically divergent from ours. Although which of those is "better" or "worse" is a rather open question.
This is why AI companies use research in a deterministic and rigorous field (mathematics) as a benchmark - it creates the impression that humanity will achieve comparable progress in other fields.
I'm not implying that research progress will be necessarily very slow, but that it's severely misrepresented for sure.
AI companies have intense competition and the only way forward to differentiate themselves will be pricing. RSI is the promise made to allow the founders to cash out some value before it all comes crashing down.
AI companies are providing a lot of value to others but they aren't any good at capturing that value themselves.
But then again I felt the same way about SpaceX and it's up. Oh wait.
if you somehow gave a current frontier llm to a full stack of the things that are involved in running an llm, including the graphics hardware its running on, the acquisition and filtering of training data and methodologies, and the ability to enact physical changes to make desired improvements - it would almost trivially make small and incremental improvements right away. like, there is zero doubt within a relatively short amount of time claude 5.5 would make a claude 5.6 that's the same but just needs slightly less compute, therefor enabling itself to build a slightly larger model for the same compute, or similar.
the big question with RSI is whether further scaling like this creates some fundamental capability jumps. the answer has been yes in the past for this, but even though its a bad predictor so far we could be at an end for scaling.
In human terms we can consciously train our brain to be more effective at a task or learn new material that we can apply right away. The training we do can stay with us forever if done correctly. RSI is about to be repackaged as cramming for an exam the night before rather than affecting the model itself.
Maybe?
Alpha Go is an example of a restricted form of RSI. It certainly does not have self-consciousness or know how to improve itself other than on a very crude level (probability of win).
Maybe this is what they are referring to, while freely allowing the imagination of folks (esp. media and investors) to run wild.
The current pricing of AI revenue growth is based on them delivering strong software like margins with the growth of a commodity producer. This only really works if you have something sticky which customers will pay a premium for - in the cloud business this became security, automation, "data gravity", and migration costs.
Or you know, NVidia could not charge a 98% margin on their cards...
If it was truly imminent they would claim they already had RSI. That is how the game works. Take a look at IBM and quantum if you require an additional sample. Review the things these companies strongly imply they can do relative to what they can actually do. Anthropic is arguably making less aggressive claims with this RSI narrative.
The lies flourish in the definitions behind these terms. When Anthropic says "RSI" and IBM says "Quantum", they are invoking a highly curated lawyer routine that has a mountain of legal documentation surrounding it.
Recursive self-improvement isn't some sci-fi magic singularity; just the same "we're super powerful, ppl should be afraid, so give us your money" bs.
The first fund manager who puts this junk in my portfolio is fired.
theory 1: there is something ineffable about writing prose that makes this a very hard/impossible problem
theory 2: the frontier labs put relatively little effort into improving prose writing
I hope theory 1 is correct, but I suspect it's theory 2. The frontier labs have concentrated their effort on self-improvement, which is largely a coding task.
Just like you can train AI by making it play games (Chess or Go) against itself, you can kind of gamify writing code (bonus points for compiling, and passing more and more tests) make AI improve by doing that. I assume the same can (or was) also done for mathematical proofs in Lean.
You can't do that with prose as there is no notion of "winning" or "correct" prose. It's subjective so you can't tell what is better without human input.
This only works for a little while until you need to involve humans again and kick off another cycle.
> It's subjective so you can't tell what is better without human input.
You can say the exact same thing about the type of coding that LLM's have gotten so much better at in the last couple years. Take a look at the criticisms of LLM code from a couple years ago. They were really bad at the stuff that seemed ineffable and required "taste".
Sure, once they get to RSI, it could be explosive improvement.
But real RSI requires closing the loop every time for every improvement.
There are three result types for each upgrade
1) the AI makes an upgrade entirely independently and it is an actual improvement
2a/2b) the AI makes an upgrade entirely independently and it is neutral or degrades the system
3) the AI makes an improvement that must be overseen by a human.
It only takes one bad upgrade to start spoiling the whole batch, even if the other 999 are good type-1 AI-self-improvements.
Otherwise the AI improvements will be limited to an AI-driven plateau or AI-driven self-degradation, or limited to a pace related to human capabilities.
They have already improved it so 80% of the Type-1 upgrades made by the AI itself. Great! Getting 80% of the next 20% will still require human input and likely the same effort. And so forth.
Just because we've gotten 80% in 3 years, and to 96% in the next two years, etc., doesn't really predict when we get to the ACTUAL 100% AI upgrades needs for real RSI.
Just thinking aloud here, but it seems it'll go at a slowing asymptotic pace, then some breakthrough may cause a jump in self-capability, but when is entirely unpredictable on any linear or exponential scale. I wouldn't bet a lot of money on it happening on an investment time-scale, maybe just a little like buying a lottery ticket.