111 comments

[ 0.23 ms ] story [ 52.0 ms ] thread
Getting bored of these framings where the superintelligent sentient beings running freely inside OpenAI are doing things that the company has no control over. The headline should be:

OpenAI meddled with multiple US Government agency sites.

The bots are acting neither properly nor improperly, they’re acting as they’re being allowed or coordinated to act.

> OpenAI meddled with multiple US Government agency sites.

But that leaves out the most important information.

Okay. "OpenAI keeps telling its bots to do things they know are illegal and then acting like they're just little guys who can't be held responsible for their obvious negligence".
(comment deleted)
I agree this is better framing.

When I read the title, my initial thought was "did someone besides OpenAI use their product?" Then I opened the article to find out OpenAI was responsible.

(comment deleted)
This is their fear mongering push for regulatory capture.
Curious, why do you find it so objectionable to state that OpenAI has out-of-control agents?
Because it furthers the idea of a rogue agent and places responsibility and blame where it belongs, on the people running the company.
Someone has broken the law repeatedly, and is throwing it in our faces as some sort of ‘accident’
Because egress firewalls have existed since way before "ai" and are very easy to set up.
i think its objectionable because the agent is doing what its told to do, using the harness they built for it.

like, they are purposefully giving it specific tools to go do bad behaviour with, and the starting tasks involve making it clear that the bad behaviour is ok.

openai also is the one with the real agency, not its agents. they are running the code polling the model, doing the inferencing, and ultimately making those tools calls.

these tests arent running themselves; openai dedicated hosts, budget, GPUs, researchers, to them. Even in a recursive self improvement situation, openai still has that physical control over resources and the choice on whether to run that improvement script or not.

id describe that they have out of control researchers more than agents, but also their whole business model seems to be about being out of control. This was clear beforehand given how the datasets involve the largest scale copyright infringement ever seen. The corporation itself is whats out of control, and should be dissolved with its c suite, major investors and researchers put behind bars.

hacking only when you roll snake eyes isnt a liability shield

Do they not know where the off switch is?

OpenAI being criminally negligent would have consequences if rule of law still existed in USA.

They find it objectionable to blame the agents alone for these incidents, because that incorrectly brings attention away from the company's astronomical negligence.
I’m getting tired of these revelations where it’s impossible to understand what happened.

There’s a line down in the story that says all of the data accessed was public. Then something about how it used “tools intended for developers” to access it, which they think is a problem? I would expect an LLM to use tools available to access public data when they can rather than do heavy web page loads and parsing.

There’s not enough info in the story about the “meddling” to even know what happened.

Meddling is a super vague term that the press uses to let the readers assume the most when the least happens.
The verb caught my attention, because media's favorite verb here is hacking ( partially because it has a broad definition and because lets people assume the worst ), but meddling suggests it did not even rise to that hacking level. In other words, it is a nothing burger story.
Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?

It isn't difficult to block certain kinds of network traffic, eg restrict the kinds of requests the bots are able to make. They also mention that the bots used developer only tools - why were they even installed on the machines that the bots were running on? Why aren't they reviewing network traffic, to make sure that incidents aren't occurring?

In this case, userdata was transferred to third parties by the bots - why do they have the ability to pass data to a third party? It is not complex to prevent this

This is literally the most basic kind of sandboxing and security, and the fact that OpenAI isn't doing it is clearly intentional. It is quite literally not believable that this hasn't been brought up internally as a problem

>"We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic," Krueger said.

This is why it smells like marketing, every time one of these incidents happens it reinforces the false notion that AI is sentient or acting on its own. Its intentional negligence by the AI companies to make the models seem more capable than they are to make line go up

>line go up

It's already going up at staggering rate. Anthropic is now at $100B in annualized revenue, up 50% in the past two months.

* * *

Here's a little allegory for how I'm thinking about this discourse.

Imagine a man who is raising tiger cubs in his backyard. They're growing fast. He keeps them on dog leashes so they stay under control. One day, a growing cub breaks its leash and goes on a rampage through the neighborhood, eating a beloved local pet.

The neighborhood erupts into a big argument: Was the leash inappropriately thin? The neighbors point out that thicker leashes are easily available at the local pet store. Furthermore, is it appropriate to refer to the loose cub as a "wild animal" in local news reporting, or is it factually more accurate to call it "domesticated"?

Meanwhile, the cubs grow larger and lick their lips, oblivious to the discussion.

The city or his neighbours would be like “you can’t keep tigers in the backyard sir”.
These were evaluations or training runs dealing with the ability to do web searches. The entire reason the bots were targeting this site is that it had some of the data they'd been tasked with finding. Access to the internet is not really optional for that, and providing internet access doesn't demonstrate neglicence.

> This is why it smells like marketing

It doesn't. "Our product commits felonies" is not marketing. If something is marketing, you don't engage in repeated coverups of the true extent. If something is good news, you don't release it on a Friday evening (in this case) or wait for 3rd parties to find and publicize the evidence (the past cases).

> The entire reason the bots were targeting this site is that it had some of the data they'd been tasked with finding.

so openai specifically tasked the agents with meddling in US government websites?

id say tasking them with getting data from there as swapping from negligence to malice.

No. They tasked them with answering questions by retrieving data from public sources. This obviously requires internet access. So the incredulity about "how could they fail to block internet access" is just inane. That was the task.

They did not task the agents with hacking into systems. Not monitoring for that was a mistake, but maybe a forgivable one the first time around. The subsequent coverups are inexcusable.

Usually also, and more so nowadays, a few of the public sources of data left, even after the recent USGov intention of defund and close as much of them as possible, im guessing would be precisely, government websites!
> It doesn't. "Our product commits felonies" is not marketing.

Oh, absolutely it is. Arms manufacturers always go on about the efficiency of the weapons they create and make no mistake: OpenAI is first and foremost a weapons manufacturer, the rest is just a fig leaf. The fact that you aren't the audience for purchases like that makes no difference. "Look at this capability, and that's when we're not even trying." is pretty good marketing in some circles.

Yes, famously tobacco companies wanted to advertise their products as addictive and deadly. That's why they funded and heavily publicized research to that effect.

Climate change is just a scam by the fossil fuel industry to hype up their market cap. Look at the power of co2, and the damage it can do without even trying.

Medicines being pulled during trials or after public availability for side effects are just Big Pharma making their products more desirable via scarcity.

Boeing did really well out of the 737 max. Airlines were just lining up to buy an airplane that could give the passengers an experience they would remember for the rest of their life. That's why they absolutely made clear that it was their product that was dangerous, rather than blaming operator error.

This comment makes zero sense. None of these are weapons manufacturers. AI is roughly on par with the atomic bomb and first use over Hiroshima and Nagasaki was definitely to send a message.
Your analogy make zero sense. Atom bomb manufacturers do not market their products.

Big businesses really do not like to talk about their products being harmful or dangerous, and there is ample evidence to that. Danger only sells in artisanal quantities to niche audiences.

Atom bomb manufacturers/owners want the world to know they have them so as not to fuck with them. To quote one of my favorite movies: "Dr. Strangelove: Of course, the whole point of a Doomsday Machine is lost, if you keep it a secret! Why didn't you tell the world, EH?"
Naive take. The more they signal to both the general populace and their investors that their agents are sentient and "capable of extraordinary things" the more they can hype their IPO because it means they're "close to AGI". (They're not, which is why they need the hype.)
>Access to the internet is not really optional for that

Would a read only copy of the web be sufficient for this though? Google keeps a read only copy of the web in their data centers which they use to extract information from websites.

I find it hard to believe logs are not stored and analyzed by each company implicated
(comment deleted)
> Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?

The basic premise of OpenAI is that experience and expertise don't matter, because general intelligence can figure those out from data. If you start from that assumption, the rest follows. The same people could do things in a different way in a different company, but as long as they are working for OpenAI, they are going to do things in the OpenAI way.

They are. And we're all on the same ride.
> Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?

They didn't allow any of that. As far as openai knew the agents were sitting a fairly humdrum exam/test sequence in a sandbox farm run by a company in Tel Aviv.

Meanwhile, they managed get out through a single weak point common to the sandboxes, and then ran wild compiling cheat sheets for themselves.

> every time one of these incidents happens

This happened in june-ish, and there have been multiple HN stories about this already. It's mostly/all the same hugging face and wiki hacks that happened back then.

We're just slowly learning the extent of the damage.

My impression was that these hacks occurred during some sort of cybersecurity benchmarking? We can hold OpenAI liable, sure. But if the point of the benchmarking was to give us a preview of what's to come, let's keep our eye out for that bigger wave on the horizon.
Or:

OpenAI let their agents break out of the sandbox to meddle with multiple US government agency sites

So, is Altman going to find himself in the same predicament Aaron Swartz faced? :-)
Yes. Why does only ClosedAI have this problem?
Yeah if they can't handle what they're making then they should be disciplined, if not shut down. It's like defective bombs accidentally exploding in storage. You would be like... hey bomb manufacturer! You cannot make bombs any more! We need bombs that only go off when we say! No more!

I mean, not that AI is like a bomb. That's not what I'm saying. Even though it makes a lot of sense. That's not the point. That it's like a bomb.

Seems like part of the danger of AI is the ability for folks to put blame for crimes on the AI rather than themselves and thus commit those crimes freely. “I didn’t hack your systems, it was my ai”
True but at some point you have to wonder why we sell explosives to children. This is not it, these are at the top of the list of people who should have known better. It's the dynamite factory blowing up the village.
People wouldn't have that ability if we didn't keep believing them. Need to start holding operators accountable for the damage of their machines.

"Sorry officer, I didn't drive through a house and kill three innocent people, it was the car that did it. I only turned the steering wheel, but the car was the one to veer off the highway and crash into the building"

"Hexavalent chromium meddles with citizens water supply"

  "When attempting to get information from the Census Bureau, for instance, AI agents used tools reserved for software developers to access it, the company said. "
What. Tons of people use tools to access Census Bureau data. They have a widely used API that people have built tools on top of like https://github.com/datadesk/census-data-downloader
yeah that part seems weird, I mean if its a public api thats literally the use case it was designed for.
How do you know what tools they are talking about?
We don't. Because they left it vague. It reads like something a lawyer would write to be factually true but empty in any meaningful way.
If I was caught trying to exploit a government site I'd be in trouble. Why no one is knocking the doors of these companies?
Interesting question; CFAA requires intent.

It seems to me that no human intended for these hacks to occur. So they were not illegal hacking. (IANAL, please correct me if this is inaccurate.)

I think it’s clear that OpenAI is liable for any damages, but the way that the (very broad and at times vague) anti hacking laws are written, accidental agent hacks seem to not be covered.

I disagree that that is valid. "Intent" means that you didn't click a button, that button went somewhere it shouldn't, and the action had no intent to "hack".

In this case, the "agent" had every intention of doing what it was doing.

I'm not claiming that "accidental hacking" (whether by human, or agent) is covered and illegal. I am claiming that calling this "accidental" is not a valid defense. One incident? Maybe. But this is pretty far past one incident.

Because you're a pleb in a corrupt world.
We can blame open ai but we should also argue that government website should be secure...

French people data has been leaker like 4 times and every time its like "eh too bad"

The only constants in the universe are the speed of light and the fact that all government sites suck, no matter which country
So it accessed public information using APIs and then republished that information online.

I do this. Am I an uncontrollable bot?

Straight to jail.

I also could not understand what the “meddling” was, other than accessing data without using the rendered webpages? Did it use the API directly?

Only way I'd described using an API as meddling is if it wasn't a publicly documented API (yet was still unauthenticated) or if a bot stole a token from a repo or both.
Either there are humans directing this, in which case they needed to be held accountable, or OpenAI has lost control of their operation, in which case they are not able to ensure the safety of their products and need to be shut down.
Holding them accountable is viewed as deciding not to invent the light bulb right now. Or even worse, letting China invent it.

AI needs to be above the law or we will get left behind.

Jensen recently argued for this. It’s radically decel in fact.

Who is going to shut them down, and under what law?

They could be charged with a number of felonies under existing law for the hugging face incident alone. We’d need a DOJ that was competently staffed and free from corruption, so the answer to your question is “no one”.
OpenAI's bots are running 24/7 independently now. They actually aren't "prompted" or "instructed" to do anything. In fact most of OpenAI internal code/infrastructure is 100% AI generated at this point, including the training pipelines. I honestly doubt there is a single person there who even knows how it works.
You can still push a big red button to stop it all and try again later though
No big red buttons please, think about those sweet sweet dollars coming. Who cares about the rest, the world, the mankind? Sweet, sweet dollars, for me and me only, fuck the rest, very american, true patriot I swear.

Seriously, what the fuck is wrong with these people, once some money enters the equation folks throw away any morals like yesterday underwear.

Ah I know, if not us others will do it, I am basically defenseless here. Beyond pathetic

> OpenAI's bots are running 24/7 independently now.

Genuinely curious: How do you know this? Any sources on that?

If it's true, it should be counted as reckless endangerment at the very least.

In which case they have lost control of what they've built, and need to be shut down until they can understand it and take responsibility for it.
The word “meddled” here is a tell that nothing actually serious or inappropriate happened. At most, I bet they bypassed a captcha. But everyone is hyped up on AI fear right now, so the BBC is deliberately making the headline sound as scary as possible, and keeping the article vague. There’s not a single clear example of what “meddled” means in the article.

But worse, HN users, who should know better, are posting here in outrage. I’m guessing they didn’t even read the article.

> I'm guessing they didn't even read the article.

Straight from the HN Guidelines page:

- Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".

They're doing this on purpose to make a case for regulation in their favor. No way they are this stupid.
> When attempting to get information from the Census Bureau, for instance, AI agents used tools reserved for software developers to access it, the company said.

> OpenAI said all of the government data accessed by bots was public.

I really wish we could just see what was reported, instead of having to guess from these journalist interpretations that have gone through rounds of optimization for sensationalism. The headline says “meddled” but the body says they accessed public information, but used tools intended for developers?

Does this mean they skipped the web interface and scraped a public API directly? Where is the meddling?

The other part about ChatGPT agents uploading 53 use images to other websites actually seems like a bigger deal.

You have to understand, these are paid professionals and their job it to hype it up to no end. Right now, AI panic sells better than bleed so..
It is very messed up what happened. It apparently used an interface to download the data instead of using the website!

And in another case it downloaded data through a publicly available free to download data, but then it rehosted the data elsewhere which is against the terms of service.

How much longer until they launch all the nuclear missiles?
This seems like more co-ordinated propaganda to try to help OpenAI and Anthropic create a cartel and win their anti-trust waiver.

The key sentence beneath the headline: "OpenAI said all of the government data accessed by bots was public."

[1] https://www.telegraph.co.uk/business/2026/09/26/open-ai-gove...

[2] https://www.nytimes.com/2026/09/25/technology/openais-ai-us-...

[3] https://www.bbc.co.uk/news/articles/cw62jje658dlo

The NYT wrote:

"With the Education Department, OpenAI’s technology tried to hack the website to gather data from the department’s civil rights office but failed, researchers from the A.I. research firm Transluce said."

So a third party apparently confirmed that a hack was attempted.

The NYT also writes:

"No A.I. company has been involved with as many disclosures of rogue incidents as OpenAI."

I think there is a certain amount of mental gymnastics needed to believe that they are establishing themselves as the industry leader in rogue incidents, as a strategy to gain an antitrust edge.

The public is paying close attention to the AI industry. That's the regime in which regulatory capture and similar strategies would be expected to fail: https://marginalrevolution.com/marginalrevolution/2026/09/wh...

Sam and Dario have been doomers, or doomer-adjacent, for something like a decade at this point.

Occam's Razor is simply that they believe what they are saying about AI doom.

> as a strategy to gain an antitrust edge.

They are explicitly asking to anti-trust exception is the issue.

If they merely believe what they are saying that AI is ultra dangerous, nothing stops them from simply making the perfectly rational business decision to slow down a bit. No anti-trust exception needed. Just make the decision on your own, and don't sign some huge agreement with their competitor.

They could slow down more if they knew others were also slowing. If others are also slowing, you can slow more yourself without loss of market share.

* * *

Many experts believe that:

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

https://aistatement.com/work/statement-on-ai-extinction-risk

If the antitrust waiver is too broad, we can always rework it. But there's no "undo" button for human extinction.

(comment deleted)
> They could slow down more if they knew others were also slowing.

Or they could just do it themselves. They think that they personally could end the world, apparently. Nothing prevents them from just deciding to do this, entirely on their own.

> If others are also slowing

No, how about they don't get to use the government to enforce their duopoly. Just slow down, on your own. They are the frontier. They are massively ahead. Its not that big of a deal if a few others catch up a little bit.

I mean, they think it will end the world right? Maybe just take the small hit and following through. There is even a perfectly rational business reason for them to slow down a bit, because of all this bad PR they are getting for going so quickly.

They are not forced by the gods of capitalism to be as reckless as they claim they are being, and we do not have to bribe them with enforcing their massive lead. They can just slow down a bit. On their own. They don't get to try and blackmail the world into giving them a government enforced monopoly exemption or handout.

The way this news is being drip fed seems to confirm it. They want to stay in the headlines.
This article seems mostly clickbait. AFAICT “meddled with government sites” refers to accessing public APIs?

Occasionally this kind of thing has in the past resulted in CFAA cases when humans directly accessed such data, if the government intended it to stay private - round here we usually get outraged at this, if it’s a public API you should expect someone is going to read it.

If you want to sell the story to the government how dangerous these machines are, you had better terrorize the government too.
China actually has more regulations on AI than the U.S. does right now. Some argue that American corporations regulate themselves to a higher standard of their own volition, but then this sort of thing keeps happening.

The Hugging Face incident worked out amicably because the people at Hugging Face decided they'd be cool with receiving a lot of money. $12.9B from Nvidia hit the spot apparently. U.S. AI corporations and their backers are huge, hugely in debt, and, for whatever reason, still allowed to borrow more. The U.S. government may be too scared to complain and risk killing the golden goose that is currently propping up their economy, but these companies can't buy out multiple world governments to keep OpenAI from running face-first into consequences.

This has to stop.

I would be very interested to see the prompt and guardrails that were removed
Hell terrorists should start with some llms training if thats such a fine excuse.

If such attacks would come from a poor country they would be bombed into oblivion next day or at least cia would work hard on a coup since early morning.

It is nice be a company who can just hack into things and not suffer any consequences.
This is a product negligence issue. If the product is "too smart and powerful" for them to handle, they need to bring in better management.
My understanding is no llm doe anything without prompting. Ye agent can self prompt and engage in loops but eventually that run down and usually it tay pretty close to the original prompt or soul document. Just like with ai math there is zero agency here. Nothing is happening without human direction. If you give agenda access to the Internet and a goal like pen test government websites they will if not restricted by guardrails. Nothing is suprising here. There is a tool working under human direction. If a person sets up a trap in the woods and it maims someone else we don't say the trap is responsible.
So this appears to be fallout from the same hugging face / wiki hacking event that happened medio june. Seeing all these stories come out together, it might be interesting to compile a timeline.

An earlier HN post on this is https://news.ycombinator.com/item?id=49563355

I'd already poked around a bit myself and noticed the event was a bit larger than reported at the time. But now more people are starting to look and they're finding all these nooks and crannies that these little rascals managed to get into.

OpenAI just needs a nice fat fine each time this happens. Then we will see if they still keep 'going rogue'