Does Anthropic currently have cybersec people able to provide a standard assessment of the kind the community expects?
This could be a corporate move as some people claim, but I wonder if the cause is simply that their talents are currently somewhere else and they don’t have the company structure in place to deliver properly in this matter.
(If that is the case they are not then free of blame, it’s just a different conversation)
This article does seem to raise some serious issues with the anthropic report. I wonder if anthropic will release proof of what they claim, or whether the report was a marketing/scare-tactic push to have AI used by defender, like the article suggests it is?
Its seems that various LLM companies try to fear monger. Saying how dangerous it is to use them in "certain ways". With the possible intention to lobby for legislation.
But what is the big game here? Is it all about creating gates to keep out other LLM companies getting market share? (Only our model is safe to use) Or how sincere are the concerncs regarding LLMs?
What would AGI actually mean for security? Does it heavily favor attackers or defenders? Even LLM, it may not help much in defense but it could teach attackers a lot right? What if employees gave the LLM info during their use that attackers could then get re-fed and study?
The lack of evidence before attributing the attack(s) to a Chinese sponsored group makes me correlate this report with recent statements from companies in the AI space about how China is about to surpass US in the AI race. Ultimately statements and reports like these seem more like an attempt to make the US government step in and be the big investor that keeps the money flowing rather than anything else.
They're an AI research company that detected misuse of their own product. This is like "Microsoft detected people using Excel macros for malware delivery" not "Mandiant publishes APT28 threat intelligence". They aren't trying to help SOCs detect this specific campaign. It's warning an entire industry about a new attack modality.
What would the IoCs even be? "Malicious Claude Code API keys"?
The intended audience is more like - AI safety researchers, policy makers, other AI companies, the broader security community understanding capability shifts, etc.
It seems the author pattern-matched "threat intelligence report" and was bothered that it didn't fit their narrow template.
Dario has been a reds scare jukebox for a while.Dario has for a year been trying to convince us how open source cCp AI bad and closed source American AI good.
Dario driven by the democratic ideals he holds dear has our best interests at heart.
Let us all support the banning of cCp's open source AI and welcome Dario's angelic firewall.
I agree so much with this. And am so sick of AI labs, who genuinely do have access to some really great engineers, putting stuff out that just doesn't pass the smell test. GPT-5's system card was pathetic. Big-talk of Microsoft doing red-teaming in ill-specified ways, entirely unreproducable. All the labs are "pro-research" but they again-and-again release whitepapers and pump headlines without producing the code and data alongside their claims. This just feeds into the shill-cycle of journalists doing 'research' and finding 'shocking thing AI told me today' and somehow being immune to the normal expectations of burden-of-proof.
When I worked at a FAANG with a "world leading" AI lab (now run by a teenage data labeller) as an SRE/sysadmin I was asked to use a modified version of a foundation model which was steered towards infosec stuff.
We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box.
It helped a little, but it wasn't all that helpful.
but in terms of coordination, I can't see how it would be useful.
the same for claude, you're API is tied to a bankaccount, and vibe coding a command and control system on a very public system seems like a bad choice.
> the same for claude, you're API is tied to a bankaccount, and vibe coding a command and control system on a very public system seems like a bad choice.
Aside from middlemen as others have suggested - You can also just procure hundreds of hacked accounts for any major service through spyware data dump marketplaces. Some percentage of them will have payment already set up. Steal their browser cookies, use it until they notice and cancel / change their password, then move on to the next stolen account. Happens all the time these days.
People grossly underestimate APTs. It is more common than an average IT curious person thinks. I happened to be oncall when one of these guys hacked into Gmail from our infra. It took principal security engineers a few days before they could clearly understand what happened. Multiple zero days, stolen credit cards, massive social campaign to get one of the Google admins click on a funny cat video finally. The investigation revealed which state actor was involved because they did not bother to mask what exactly they were looking for. AI just accelerates the effectiveness of such attacks, lowers the bar a bit. Maybe quite a bit?
This is an excellent article. Anthropic's "paper" is just rambling slop without any details that inserts the word "Claude" 50 times.
We have arrived at a stage where pseudoscience is enough to convince investors. This is different from 2000, where the tech existed but its growth was overstated.
Tesla could announce a fully-self-flying space car with an Alcubierre drive by 2027 and people would upvote it on X and buy shares.
I was at an AI/cybersecurity conference recently and the talk given by someone from Anthropic was a lot like this report: tantalizing, vague, and disappointing. The speaker alluded to similar parts of this report. It was though everything was reflected through Claude, simultaneously polished, impressive, and lost in the deep end.
"Look, is it very likely that Threat Actors are using these Agents with bad intentions, no one is disputing that. But this report does not meet the standard of publishing for serious companies."
Title should have been, "I need more info from Anthropic."
I'm baffled at the assumption that concrete and specific evidence of international (presumably) hostile espionage that is currently being enacted using X and Y software and Z specific techniques, would be publicly released in real time.
I can't think of a single situation in which it would be reasonable to assume that.
It's not like we even get governments or corporations saying 'oh hey, just raising the alarm that bad people are using this Photoshop feature to create fake cheques which they're then depositing into their accounts, so bank staff, be on the lookout!'
Because yeah, that's a Photoshop ad.
And it's not like espionage is new, like the Chinese side have been ramping up for decades now, or like there has ever been an expectation that companies with suspicions or evidence of international subterfuge should... should lay it all out in a public report? Is that really what the article is expecting?
I don't even think the UK has got around to officially acknowledging Funny Business in UK-Argentinian relations in any documents or events during the 80's, and the secret was rather given away around the time we went to all out literal war.
We know things must have built up before the day war was declared, but nobody expected every escalation of diplomatic unrest to be communicated to the entire nation in real time.
Because that would be deranged.
Idk, maybe I'm misunderstanding something about the article. I feel like it isn't in my field, although I'm not entirely sure what field specific knowledge I'm missing to make sense of this.
I would very much like to agree with the sentiment, I'm always down for some AI-dissing and a bit of tin foil hat Big Tech Analyses.
But I couldn't get much more than "This company is lying because it didn't give me any Chinese State secrets, let alone explain how to get stars secrets using their software,' which feels so censored as to be pointless, or just kinda wildly petty and ill informed
With the Wall Street wagons circling on the AI bubble expect more and more puff PR attempts to portray “no guys really, I know it looks like we have no business model but this stuff really is valuable! We just need a bit more time and money!”
79 comments
[ 3.1 ms ] story [ 79.9 ms ] threadThis could be a corporate move as some people claim, but I wonder if the cause is simply that their talents are currently somewhere else and they don’t have the company structure in place to deliver properly in this matter.
(If that is the case they are not then free of blame, it’s just a different conversation)
But what is the big game here? Is it all about creating gates to keep out other LLM companies getting market share? (Only our model is safe to use) Or how sincere are the concerncs regarding LLMs?
Someone make this make sense.
They're an AI research company that detected misuse of their own product. This is like "Microsoft detected people using Excel macros for malware delivery" not "Mandiant publishes APT28 threat intelligence". They aren't trying to help SOCs detect this specific campaign. It's warning an entire industry about a new attack modality.
What would the IoCs even be? "Malicious Claude Code API keys"?
The intended audience is more like - AI safety researchers, policy makers, other AI companies, the broader security community understanding capability shifts, etc.
It seems the author pattern-matched "threat intelligence report" and was bothered that it didn't fit their narrow template.
I agree so much with this. And am so sick of AI labs, who genuinely do have access to some really great engineers, putting stuff out that just doesn't pass the smell test. GPT-5's system card was pathetic. Big-talk of Microsoft doing red-teaming in ill-specified ways, entirely unreproducable. All the labs are "pro-research" but they again-and-again release whitepapers and pump headlines without producing the code and data alongside their claims. This just feeds into the shill-cycle of journalists doing 'research' and finding 'shocking thing AI told me today' and somehow being immune to the normal expectations of burden-of-proof.
We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box.
It helped a little, but it wasn't all that helpful.
but in terms of coordination, I can't see how it would be useful.
the same for claude, you're API is tied to a bankaccount, and vibe coding a command and control system on a very public system seems like a bad choice.
Aside from middlemen as others have suggested - You can also just procure hundreds of hacked accounts for any major service through spyware data dump marketplaces. Some percentage of them will have payment already set up. Steal their browser cookies, use it until they notice and cancel / change their password, then move on to the next stolen account. Happens all the time these days.
We have arrived at a stage where pseudoscience is enough to convince investors. This is different from 2000, where the tech existed but its growth was overstated.
Tesla could announce a fully-self-flying space car with an Alcubierre drive by 2027 and people would upvote it on X and buy shares.
Instead of accusing of China in espionage perhaps they have to think about why they force their users to use phone numbers to register.
"Look, is it very likely that Threat Actors are using these Agents with bad intentions, no one is disputing that. But this report does not meet the standard of publishing for serious companies."
Title should have been, "I need more info from Anthropic."
I can't think of a single situation in which it would be reasonable to assume that.
It's not like we even get governments or corporations saying 'oh hey, just raising the alarm that bad people are using this Photoshop feature to create fake cheques which they're then depositing into their accounts, so bank staff, be on the lookout!' Because yeah, that's a Photoshop ad.
And it's not like espionage is new, like the Chinese side have been ramping up for decades now, or like there has ever been an expectation that companies with suspicions or evidence of international subterfuge should... should lay it all out in a public report? Is that really what the article is expecting?
I don't even think the UK has got around to officially acknowledging Funny Business in UK-Argentinian relations in any documents or events during the 80's, and the secret was rather given away around the time we went to all out literal war. We know things must have built up before the day war was declared, but nobody expected every escalation of diplomatic unrest to be communicated to the entire nation in real time. Because that would be deranged.
Idk, maybe I'm misunderstanding something about the article. I feel like it isn't in my field, although I'm not entirely sure what field specific knowledge I'm missing to make sense of this.
I would very much like to agree with the sentiment, I'm always down for some AI-dissing and a bit of tin foil hat Big Tech Analyses.
But I couldn't get much more than "This company is lying because it didn't give me any Chinese State secrets, let alone explain how to get stars secrets using their software,' which feels so censored as to be pointless, or just kinda wildly petty and ill informed
With the Wall Street wagons circling on the AI bubble expect more and more puff PR attempts to portray “no guys really, I know it looks like we have no business model but this stuff really is valuable! We just need a bit more time and money!”