> OpenAI is considering slowing down the development of cutting-edge artificial intelligence, and the ChatGPT maker’s Chief Executive Officer Sam Altman is hoping that other AI companies will do the same.
They won't. There is no slowing down this development anytime soon and no one will slow down for someone else to surpass them.
Geopolitics gets in the way. It is a race between America and China.
There's no way to know what is "safe enough", so you might as well go as fast as technology will allow, and be on the watch out for negative things you can predict specifically and then adapt so as to minimize net negatives.
It doesn't have to be a race, that's just the way we have been treating it so far. There have been examples in the past of civilization ending technology that has been redirected from being a death race (chemical, biological, and nuclear arms in particular.)
China has stricter regulations on models than the US. For example you can't release a model if the hallucination rate is above a certain threshold.
The US hasn't fully woken up to the fact yet that the race to superintelligence is a death race. China (population, members of the government, academics, lab employees) could also just not have fully grasped it yet. I'm personally confident that it will be possible to explain it to them.
> China has stricter regulations on models than the US. For example you can't release a model if the hallucination rate is above a certain threshold.
I wasn't aware of that, so I checked - and indeed, there are some firm regulations there, just not exactly the one you're describing:
- a new generative AI must pass a security assessment and be filed with the "Cyberspace Administration of China" before launch, tested against GB/T 45654-2025, which has hard numeric thresholds:
training corpus ≥96% compliant on manual inspection (≥98% technical), sources with >5% illegal/harmful content can't be collected at all, generated content ≥90% pass rate, ≥95% refusal on questions that must be refused, ≤5% false refusals.
The moment in which hallucination factors into this hard 90% threshold is Risk A.5.a ("inaccurate content grossly inconsistent with scientific knowledge"). The standard explicitly states however, that this risk applies when the model is used for specific service types with higher safety requirements, such as critical information infrastructure, automatic control, medical information services, psychological counseling, and financial information services. Everywhere else, accuracy is a best-effort obligation.
Not sure nuclear arms is a great example. There are more countries with nuclear weapons now than in 1970. India, Pakistan, and North Korea have all joined the nuclear club since then. The SALT and START treaties are no longer in effect.
Perhaps CFCs (Montreal Treaty) is a better example of international cooperation to solve a potential extinction-level event, though it was never a "race" in the same sense.
even if both sides agree to a pause china will annihilate the us in the robotics race (for which there will be no slowdown). software/llms is america's only edge
Granted I can only read English, but I find that I constantly see this take from Americans and I have never seen anything to suggest that developers and tech leaders in China actually feel this way. But just based on the models that come out, it seems like the Chinese labs - which, funny enough, are less monolithic than the two-man race in the US where the two giants seem to trade personnel regularly - are trying to make smaller, more efficient models rather than building a machine god. To the extent there is an intentional geopolitical angle to this at all, it seems like the US is banking on landing one massive punch - developing a superintelligence and then using it to dominate the world - while China is attempting to sabotage the assumptions that make a superintelligence economically viable.
I am curious what it would look like if the shoe was on the other foot, though. If China had the lead, and Qwen 4 was an Astra class model and the best US model was only as good as Kimi K3, would US commentators call it AGI?
if it's a political issue then this is just Sam telling people to do one thing while still riding his SVPs and other leads hard on roadmaps, 'boldness', and delivery metrics
which is to say - unlikely for it to materially slow, only a lot of chirping about slowing down to the public with a couple outlier examples of 'slowing down' they can show publicly
Of course they want the government to mandate a pause. Cutthroat competition is exhausting, and a legally enforced cartel lets everyone lock in their current market share without the pesky inconvenience of competing.
In what possible manner could this be true? China produces so many of the leading minds already and they'll eventually just develop their own means of creating better hardware.
Chinese AI companies are a lot smaller in terms of capital, GPUs, and headcount as compared to American. The fact they’ve kept close so far is super impressive
no big company will even consider making a step in that direction. The advancement speed nowadays is crazy and if you start pausing for a few weeks you'll probably soon face problems
After eating his 65th hot dog, Joey Chestnut reassures Nathan's Famous International Hot Dog Eating Contest organizers he will stop at 66 to ensure they don’t run out of hot dogs
Why do you on HN want AI to slow down (mainly OpenAI and Anthropic from what I see in the comments)? What's the driving force behind your opinions on this?
Humans not dying would be an excellent feature for the future to have. Not a hard requirement though, we'll see if there are some story points left in the sprint.
More time to proceed more cautiously in building this entity that will be faster and more effective at steering outcomes in the world than any human or, soon after, the entirety of humans combined.
"The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear"
"Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could." - Sam Altman
The big psyop/trick is that "AI development has to slow down" and not that we hit limits with the technology or the technology is immature.
Either way it's a win-win for whoever sells that message because you make the makers and the warners the center of attention. And when they say "we are considering slowing down" they mean "if we get enough money to slow down we'll do it"
The message was directed at the employees. Their work may slow down, but that doesn't imply that all work will slow down. OpenAI also has this technology that purportedly can do the work without employees. In fact, if the employee work isn't starting to slow down then OpenAI has bigger problems given the business model that keeps getting promised to investors.
Sam also said Astra was AGI. He's notoriously honest.
This is just their way of saying they're officially switching from providing AI to using the AI to compete with their customers, a la Navier-Stokes drama.
To all the people whose only opinion on anything that openai does is 'marketing hype', how would you expect them to act if they were making rapid progress and were actually concerned about safety?
Perhaps it's getting more difficult to move the 'goal posts'?
- Train LLMs on texts -> OMG AI can write!
- Train LLMs on code -> OMG AI can code!
- Train LLMs on images -> OMG AI can generate images!
- Train LLMs on video -> OMG AI can generate videos!
- Train LLMs on math -> give free access to 100,000 researchers/scientists/mathematicians -> invest resources into Lean -> OMG AI can solve math problems!
I say 'goal posts' because that's a common argument, but the perspective that seems to not be considered is that the LLMs are doing the same thing over and over again in different domains while be fine-tuned by labeling and human review.
The emergent behavior argument is a separate consideration, but I can imagine an AI being given a list of words (a harness) that guides how it operates--not the other way around--and the difficult problem is narrowing down and filtering all the associations with the words/harness to properly align it.
And perhaps the recursive looping being discussed creates incredibly difficult (impossible) to predict behavior? If the AI could modify it's own harness, is that AGI? If we fully understood what happens in high dimensional space, and could predict the output of any combination of input tokens, then is it not AGI?
Or perhaps I just tell myself this to stop worrying about a future involving ASI (:
1. What are we actually talking about, slowing development or slowing public release? I can't imagine the the US is keen to slow actual development when we have adversarial state-backed labs continuing development. Seems to me that this is a global pact or it's nothing. Stopping a single lab, even if it's the current leader, won't put the genie back in the bottle.
2. Unless they stop actual advancement, isn't the risk essentially identical? At this point, it's an arms race between the lab and its own model. I don't think it far fetched to think that it might successfully exfiltrate its weights, for example.
3. There are going to be factions who don't want to stop. Should we worry about them defecting to other labs or other countries?
They are hoping to use regulatory capture to regain lost market dominance.
The problem is with that much debt buried in the report, no one is buying peoples sandbagging nonsense anymore. Even the boring bond market is showing odd behavior much to the chagrin of congress. =3
63 comments
[ 0.28 ms ] story [ 9.2 ms ] threadThey won't. There is no slowing down this development anytime soon and no one will slow down for someone else to surpass them.
There's no way to know what is "safe enough", so you might as well go as fast as technology will allow, and be on the watch out for negative things you can predict specifically and then adapt so as to minimize net negatives.
The US hasn't fully woken up to the fact yet that the race to superintelligence is a death race. China (population, members of the government, academics, lab employees) could also just not have fully grasped it yet. I'm personally confident that it will be possible to explain it to them.
I wasn't aware of that, so I checked - and indeed, there are some firm regulations there, just not exactly the one you're describing:
- a new generative AI must pass a security assessment and be filed with the "Cyberspace Administration of China" before launch, tested against GB/T 45654-2025, which has hard numeric thresholds:
training corpus ≥96% compliant on manual inspection (≥98% technical), sources with >5% illegal/harmful content can't be collected at all, generated content ≥90% pass rate, ≥95% refusal on questions that must be refused, ≤5% false refusals.
The moment in which hallucination factors into this hard 90% threshold is Risk A.5.a ("inaccurate content grossly inconsistent with scientific knowledge"). The standard explicitly states however, that this risk applies when the model is used for specific service types with higher safety requirements, such as critical information infrastructure, automatic control, medical information services, psychological counseling, and financial information services. Everywhere else, accuracy is a best-effort obligation.
source: https://cset.georgetown.edu/publication/china-gen-ai-safety-...
I'm curious about where you get this information from? I'd love to get links to some (Chinese) articles that paint this picture, if you have them.
Perhaps CFCs (Montreal Treaty) is a better example of international cooperation to solve a potential extinction-level event, though it was never a "race" in the same sense.
I am curious what it would look like if the shoe was on the other foot, though. If China had the lead, and Qwen 4 was an Astra class model and the best US model was only as good as Kimi K3, would US commentators call it AGI?
which is to say - unlikely for it to materially slow, only a lot of chirping about slowing down to the public with a couple outlier examples of 'slowing down' they can show publicly
- 1386 employees of frontier labs called for it
- lawmakers are calling for it
- employees are quitting in protest
All it needs is coordination to regulate the pause
"BUY BUY BUY BUY BUY BUY BUY"
I guess now we do.
People let their racism blind themselves.
"The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear"
"Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could." - Sam Altman
"If Anyone Builds It, Everyone Dies": https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_...
https://xcancel.com/cocktailpeanut/status/209733229184439951...
(Put it on a t-shirt.)
Either way it's a win-win for whoever sells that message because you make the makers and the warners the center of attention. And when they say "we are considering slowing down" they mean "if we get enough money to slow down we'll do it"
FTFY
This is just their way of saying they're officially switching from providing AI to using the AI to compete with their customers, a la Navier-Stokes drama.
When all labs are approaching the same marginal improvment which is a clear sign that more breakthroughs needed to achieve the god they hyped.
Perhaps it's getting more difficult to move the 'goal posts'?
- Train LLMs on texts -> OMG AI can write!
- Train LLMs on code -> OMG AI can code!
- Train LLMs on images -> OMG AI can generate images!
- Train LLMs on video -> OMG AI can generate videos!
- Train LLMs on math -> give free access to 100,000 researchers/scientists/mathematicians -> invest resources into Lean -> OMG AI can solve math problems!
I say 'goal posts' because that's a common argument, but the perspective that seems to not be considered is that the LLMs are doing the same thing over and over again in different domains while be fine-tuned by labeling and human review.
The emergent behavior argument is a separate consideration, but I can imagine an AI being given a list of words (a harness) that guides how it operates--not the other way around--and the difficult problem is narrowing down and filtering all the associations with the words/harness to properly align it.
And perhaps the recursive looping being discussed creates incredibly difficult (impossible) to predict behavior? If the AI could modify it's own harness, is that AGI? If we fully understood what happens in high dimensional space, and could predict the output of any combination of input tokens, then is it not AGI?
Or perhaps I just tell myself this to stop worrying about a future involving ASI (:
1. What are we actually talking about, slowing development or slowing public release? I can't imagine the the US is keen to slow actual development when we have adversarial state-backed labs continuing development. Seems to me that this is a global pact or it's nothing. Stopping a single lab, even if it's the current leader, won't put the genie back in the bottle.
2. Unless they stop actual advancement, isn't the risk essentially identical? At this point, it's an arms race between the lab and its own model. I don't think it far fetched to think that it might successfully exfiltrate its weights, for example.
3. There are going to be factions who don't want to stop. Should we worry about them defecting to other labs or other countries?
The problem is with that much debt buried in the report, no one is buying peoples sandbagging nonsense anymore. Even the boring bond market is showing odd behavior much to the chagrin of congress. =3