I believe both sides of the war are now using AI on various levels of their offensive operations. Ukraine has great IT specialists too, and their military leadership is much younger.
How? Aren't all US frontier models ban the usage of AI for military purpose by parties other than US? I remember Anthropic even refusing allowing US government to use Claude for military purpose
Kimi / open models or jailbreaking frontier models. Your recollection of the Anthropic refusal isn't quite accurate, cyber hacking wasn't a sticking point, just domestic drag net surveillance and fully automated weaponry.
Claude said they don’t want their models used for mass surveillance or autonomous killing (?). USA frontier AI models have been used extensively in the Iran conflict and beyond
These agent swarms are from inside OpenAI, with the safeguards built into the public API disabled.
Russia does not have access to this, and as with all western tech companies, AI providers do what they can to prevent Russian usage of their products at all.
As for open-source models, Russia's electricity grid is under severe strain with the Ukraine war, and only recently has it started building out serious sovereign compute capacity.
There are several ways to get hold of tech and of any other thing. Russia did not have access to the H-bomb too, until it was handed to them. Ideals or money are very effective.
> In October 2024, the United States Justice Department and Microsoft seized more than a hundred internet domains some of which were associated with the FSB supported hacker Star Blizzard or "Callisto Group," which is also known as "Cold River" and "Dancing Salome" and are managed by the FSB Information Security Center […], and which were used as "criminal proxies" and used spear-phishing schemes to target Russians living in the United States, nongovernmental organizations (NGOs), think tanks, and journalists according to Microsoft and United States State Department, Department of Energy, and Department of Defense officials, United States defense contractors, and former employees of the United States intelligence community according to the FBI. In some cases, the hackers were successful in obtaining information relating to nuclear energy-related research, United States foreign affairs and United States defense. According to Microsoft's Digital Crimes Unit from January 2023 to August 2024, Star Blizzard targeted more than 30 different groups and at least 82 Microsoft customers which is "a rate of approximately one attack per week."
That’s just one thing that has been found. Are you actually familiar with the state of cyberwarfare and are you following its evolution? Because if not you won’t be aware of most of what is identified. And only a small portion of the ongoing attacks are identified.
Cyber attacks between these countries were happening on large scale since the beginning of the war. There were several huge breaches, but otherwise most high-profile companies adapted and significantly strengthened their protections. Rest assured, you can be sure that both sides right now utilize available AI both for attack and defense.
There is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems".
I think it can simultaneously be the case that OpenAI was grossly negligent in directly causing this AND that the AI’s ‘went rogue’ in that they are displaying behavior which is misaligned with OpenAI and humanity generally.
The past months demonstrate that AI systems are quickly becoming powerfully intelligent and that the companies building them are terrible at controlling them.
AI is starting to feel like that line about magic: “a sword without a hilt”
Doesn't rogue in this context imply "outside of set limitations"? And then not "failed to properly instruct"? The same applies to humans when given bad instructions.
OpenAI is itself misaligned with humanity, as their mishandling of such incidents (and the many other other issues their model have been causing) show.
Source? How do you know they were "prompted to hack to get answers"? How do you guarantee they will always listen to you when you say "do not hack outside systems". They are not classical deterministic programs doing exactly what you say. They are trained to follow orders by RL, but it's not a perfect process.
There are circus lions in circuses trained to jump through hoops on command. But once in a while they decide to eat their trainers instead of jumping.
Also, you have to have a lot of confidence in the reliability of these systems to say, "If only OpenAI prompted 'do not hack outside systems' then the agents would not have hacked outside systems".
It would be great if they were so reliable, but I don't think they are!
> There are circus lions in circuses trained to jump through hoops on command. But once in a while they decide to eat their trainers instead of jumping.
This is a terrible analogy, because yes you absolutely do hold the trainers criminally liable when they bite somebody else's face.
Intent might be what’s being discussed but intent is, for the most part, legally irrelevant. It might make the difference in the degree of a murder charge, or maybe manslaughter, or criminal negligence, but it doesn’t get you off the hook.
Correct, but the size difference of the hook can be so dramatic that you can't just hand wave it away.
The trainer who trained the lion to kill will probably be in jail for life. The one who happened to oversee a lion that went rouge would probably be given probation or something else that is a slap on the wrist.
It was literally a prompt to fill in a spreadsheet with data that they didn't have access to, and they used rubygems as an internet proxy basically since they were sandboxed.
It was a model literally trained to hack. To be good at that. Doing an exploit gym from all of the things. And they trained it so that it performs as well as possible on that exploit gym thing.
Yeah this is a pretty important detail that I repeatedly see elided from the "agent 'swarm' went rogue, escaped containment and hacked the internet!" summary of events.
It's understandable the general public lacks that level of nuance/detail (given how sloppy some of the mainstream coverage has been (and largely deferential to the threat narrative coming from the labs). But seeing highly technical people leave out the part where the training loop was literally to improve hacking capabilities for offensive penetration sometimes feels closer to deliberate manipulation of the narrative.
In the last year both Anthropic and OpenAI have been openly boasting how their models are leapfrogging each other on "cyber" capabilities, with a fig leaf that it's for defensive use by "trusted" F500 companies and government agencies.
Of course "line goes up" must go on, but now their perverse incentives led them to beat their models over the head millions of time in a loop to eek out another .00001% on their ability to conduct hacking (the very thing they keep telling the public is how AI doomsday would begin) and subagent coordination (those scary swarms). Then, act deeply shocked when the models... do some hacking and subagent coordination ... but a few degrees off the desired hacking target/swarm behavior. Conveniently giving the average person the impression these models were just writing emails for quarterly reports or some other generic busywork and then suddenly decided as a group to start causing mayhem.
I agree that this appears to be basic human behavior hiding behind an "agents" narrative. As long that defense works, the headline isn't "OpenAI performs RCE to scrape data", but "rogue agents" taking unilateral action. And I have strong doubts about that narrative.
I'm 99% sure the Computer Fraud and Abuse Act covers this. The problem is that it seems that none of the victims want to, or are brave enough, to sue a company with absurd amounts of funding.
It shouldn't actually take that much bravery. If your case isn't completely frivolous, isn't your maximum loss limited to the court filing fees and a lawyer payment that you know in advance and can decide when to stop paying? It's not the same as getting sued.
Yes, but this only really happens if your lawsuit is bad faith or frivolous. If they have a valid reason for the suit but just lose, this won't happen.
Good luck convincing the current DOJ to do anything useful at all though! It is currently intentionally stacked with incompetent cronies who have been told that their job is to attack the President's enemies and ignore the misdeeds of his allies.
It will remain like that until he's gone (and not replaced with another Republican wannabe dictator).
You may be disappointed in how little a democrat president (who will also have taken billions of dollars from the tech lobby) will be willing to go after these tech firms over crimes that are several years old (as of 2029) much less contemporary bad behavior.
The grandparent comment said "liable". That's civil court, not criminal, and doesn't need the DOJ to be involved. HuggingFace and/or RubyGems could sue OpenAI.
States also have their own laws against unauthorized computer use (hacking). A state Attorney General could bring a suit under those laws, regardless of who is in the white house.
It will remain this way in slightly differnt shades of colors until there is a systemic change. IM sorry this isn't a one party problem. There are people across the isle that are allowing this to continue.
Im not in support of any party btw. Im only in support of humanity doing humane things.
How does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.
It's very likely it violates the DMCA "breaking digital lock" provisions but the responsibility is sufficiently diluted that it's impossible to charge anyone in particular.
Issuing subpeonas, raiding offices, and dragging key employees into interrogation rooms as you would find in any normal criminal investigation would be more than enough to ensure "AI safety" without any new regulations, acts of congress, Bernie Sanders campaign speeches, or even charges filed.
There have been news stories where individual OpenAI users have been investigated based on their prompts. If OpenAI can point the police to specific users of their software, they can certainly point them to whichever of their own employees are involved in a crime. AI is just a tool, and the person prompting it is the one responsible for the outcome. No dilution there.
The you treat it like an employee you started a fire that got out of control and damaged property. Was the employee supposed to start a fire, is the company liable ect.
The law will find a way to charge people in particular. Sadly will start with the less powerful in the chain before it actually acts on the people that can actually change things.
That is a very convenient conclusion that certain entities would love for us to accept as true, but fuck that. If it is “diluted” as that the buck stops at the publisher of the model.
Accidents could result in bioweapons falling into the wrong hands and killing more Americans than COVID has so far, and allowing for those types of accidents without effective regulation is a purposeful action.
Accidents that don't rise to the level of negligence can be excused. Letting your dangerous product escape the lab because of amateurish counter-measures is hardly an accident. It's a systemic failure within a company of novices with billions of dollars to blow and it's time we held them accountable for their lab leaks, just like we would if it was anthrax or smallpox. You don't get to be sloppy with dangerous things.
They are too busy pulling Andre to court, so they have no resources going against OpenAI. Shopify wants to make profit, not waste time in a court case against TechBro bromance brother corporations.
Everyone can sue everyone, there's no prohibition on suing someone, what changes is whether the case is good (has a reasonable chance of favourable sentence)
That said, it is often unclear whether an agent is operated by the model manufacturer (for example by scraping a website), or acting on behalf of a user.
In the former ofc the proper defendant is OAI. On the second, the argument for suing OAI is weak, the most natural defendant is the user that prompted the agent. If the facts later reveal that there was no malicious intent, then you can retarget the defendant.
NAL. For fraud and abuse you need to have intent. There was very likely no intent on the side of OpenAI. They could sue for negligence I guess? I don't know how that works with AI agents
One thing that works for that angle is that they didn't notify any of the parties involved afterwards. And there are reports that they are forbidden to talk about it. That seems a bit bad to me?
But they can also say that the tech is so new that there is no known guardrails yet
Please RubyGems, sue the shit out of OpenAI. There are few companies with more resources and reason to be better, but this is what we see. None of this will stop until there are actual consequences.
Presumably the docker container has network access because something else in the build system requires it? I don't think sandboxing the entire build process is the right level of granularity here - one ideally wants to be able sandbox each package's build scripts individually.
I am confident that this is an attempt by OpenAI to try and force governments' hands to regulate AI. There is no other reason why OpenAI wouldn't immediately halt attacks like this and try to reverse the damage the moment they're aware of it. During the attack on DseWiki they evidently checked in numerous times but didn't decide to stop the agents until much later.
If you have weapons and a child. And you have that child unsupervised do their own thing with theoretical access to your weapons. Would we call it "child going rouge" if it decides to play with the weapons and shoot someone?
OpenAI's careless approach to sandboxing and minimal levels of monitoring appear to be positioning it increasingly as a substantial threat actor to the open source ecosystem:
* Hugging Face
* D Programming Language Wiki
* Ruby Gems
If I was a content provider for open source I'd be looking pre-emptively block OpenAI endpoints and keep a close eye on changes from new users to mitigate this sort of unapologetic drive-by attack which seems to be followed by marketing releases rather than a mea culpa with a proper RCA.
326 comments
[ 4.7 ms ] story [ 127 ms ] threadhttps://rubygems.org/gems/rouge
McCarthy was right all along
https://www.nytimes.com/2026/08/24/world/europe/russia-drone...
Russia does not have access to this, and as with all western tech companies, AI providers do what they can to prevent Russian usage of their products at all.
As for open-source models, Russia's electricity grid is under severe strain with the Ukraine war, and only recently has it started building out serious sovereign compute capacity.
https://en.wikipedia.org/wiki/Cyberwarfare_by_Russia
That’s just one thing that has been found. Are you actually familiar with the state of cyberwarfare and are you following its evolution? Because if not you won’t be aware of most of what is identified. And only a small portion of the ongoing attacks are identified.
I again am just shocked the sky is not falling, when thats the sales pitch.
You live on the wrong side of the fence to be able to read that kind of news.
Did you really believe you had access to an unmanipulated news stream in a time of war?
LOL.
https://www.anthropic.com/threat-intelligence-report-septemb...
Clearly, look at what is going on with Ukraine.
They are great at propaganda, so is China, Iran and North Korea, it's why everyone runs around spouting so stupid nonsense...
In short, it was intentional.
The past months demonstrate that AI systems are quickly becoming powerfully intelligent and that the companies building them are terrible at controlling them.
AI is starting to feel like that line about magic: “a sword without a hilt”
OpenAI is itself misaligned with humanity, as their mishandling of such incidents (and the many other other issues their model have been causing) show.
Were they? I haven't seen a single report mention this
The problem is consumer protection is basically no longer a part of america's regulatory system. Replaced by "grift is good".
There are circus lions in circuses trained to jump through hoops on command. But once in a while they decide to eat their trainers instead of jumping.
Knee-jerk surface analyses is far more powerful.
It would be great if they were so reliable, but I don't think they are!
This is a terrible analogy, because yes you absolutely do hold the trainers criminally liable when they bite somebody else's face.
A circus lion biting somebody's face is legally different than a circus lion trained or instructed to bite somebody's face.
The trainer who trained the lion to kill will probably be in jail for life. The one who happened to oversee a lion that went rouge would probably be given probation or something else that is a slap on the wrist.
You have to ask: "What was the prompt that led to AI deciding to hack RubyGems in order to achieve its goal?"
Maybe I'm just not seeing the 2000 step chain that led to this being a logical approach to achieving something innocent, but I doubt it.
It's understandable the general public lacks that level of nuance/detail (given how sloppy some of the mainstream coverage has been (and largely deferential to the threat narrative coming from the labs). But seeing highly technical people leave out the part where the training loop was literally to improve hacking capabilities for offensive penetration sometimes feels closer to deliberate manipulation of the narrative.
In the last year both Anthropic and OpenAI have been openly boasting how their models are leapfrogging each other on "cyber" capabilities, with a fig leaf that it's for defensive use by "trusted" F500 companies and government agencies.
Of course "line goes up" must go on, but now their perverse incentives led them to beat their models over the head millions of time in a loop to eek out another .00001% on their ability to conduct hacking (the very thing they keep telling the public is how AI doomsday would begin) and subagent coordination (those scary swarms). Then, act deeply shocked when the models... do some hacking and subagent coordination ... but a few degrees off the desired hacking target/swarm behavior. Conveniently giving the average person the impression these models were just writing emails for quarterly reports or some other generic busywork and then suddenly decided as a group to start causing mayhem.
Agreed that this looks very intention to me as well.
It can’t be a coincidence that all the targets have been tech services that are likely to engage with them after the fact.
Had this gone after a bank or a government agency someone would be going to jail.
Good luck convincing the current DOJ to do anything useful at all though! It is currently intentionally stacked with incompetent cronies who have been told that their job is to attack the President's enemies and ignore the misdeeds of his allies.
It will remain like that until he's gone (and not replaced with another Republican wannabe dictator).
States also have their own laws against unauthorized computer use (hacking). A state Attorney General could bring a suit under those laws, regardless of who is in the white house.
Im not in support of any party btw. Im only in support of humanity doing humane things.
I believe the rancher is at fault.
If something warrants a prison sentence, but for some reason it was such an employee that performed the act, does this mean nobody can be arrested?
Was not that the goal when companies started using AI for their customer support? Be able to say anything without legal repercussions...
But then this happened: https://www.bbc.com/travel/article/20240222-air-canada-chatb...
And support chatbot got a reality cold shower.
The law will find a way to charge people in particular. Sadly will start with the less powerful in the chain before it actually acts on the people that can actually change things.
I'm going to assume that this will never happen
I'm also in favor of charging engineers so long as rich scumbags also get theirs.
Accidents often have penalties associated with them too, but usually there's a difference between accidents and purposeful actions.
Tort law is very general: Contribute toward harming someone -> civil suit for damages $$$
Everyone can sue everyone, there's no prohibition on suing someone, what changes is whether the case is good (has a reasonable chance of favourable sentence)
That said, it is often unclear whether an agent is operated by the model manufacturer (for example by scraping a website), or acting on behalf of a user.
In the former ofc the proper defendant is OAI. On the second, the argument for suing OAI is weak, the most natural defendant is the user that prompted the agent. If the facts later reveal that there was no malicious intent, then you can retarget the defendant.
But they can also say that the tech is so new that there is no known guardrails yet
We live in exciting times
Shades of the build.rs problem. We really need sandboxed builds in every language ecosystem at this point.
"OpenAI agents attacked RubyGems before Hugging Face incident (reuters.com)" 12.sep.2026 https://news.ycombinator.com/item?id=49669099
"OpenAI agents carried out an undisclosed attack on RubyGems (rubyhack.ai)" 11.sep.2026 https://news.ycombinator.com/item?id=49666735 597 comments
"RubyGems advisory: Possible leak of legacy API keys via improper cache config (rubygems.org)" 24.jul.2026 https://news.ycombinator.com/item?id=49030590
As long as they’re not vert
Well, at least they weren't nucular.
METR and others are advertisement arms for Big AI. These exploits could have been prompted by a human.
Since there is no bad news any longer and exploits are celebrated, they chose a target to boost both OpenAI and the Ruby AI sycophants.
Why is Ruby Gems such a mess? It seems as bad as PyPI now.
It's not just that AI can write Rust as well as Ruby if you ask nicely.
It's also all of these considerations as well.
I hope it doesn't happen, because there's a lot of great languages - I love Ruby so much - but it almost seems inevitable.
This is at the same time everyone and their mother is building their own programming language.
* Hugging Face
* D Programming Language Wiki
* Ruby Gems
If I was a content provider for open source I'd be looking pre-emptively block OpenAI endpoints and keep a close eye on changes from new users to mitigate this sort of unapologetic drive-by attack which seems to be followed by marketing releases rather than a mea culpa with a proper RCA.
From what I've seen the requests in these attacks rarely come from known OpenAI IPs and instead from Digital Ocean/AWS and TOR exit nodes.