These guys are now the biggest slowdown to fixing issues with their bullshit cyber detection. They drop a model, let the model find security issues in your code, then when you want to fix them you need to play a game of charades so it doesn't realize it's creating a validation and then a fix for the issue. Meanwhile attackers are using some ablated deepseek or what have you.
It's frustrating. Claude thinks it finds a way to hop from one client to another, but any attempt to get it to demonstrate it isn't relying on something that's already prevented runs into "safety" concerns and gets blocked.
I'm not that hot on the idea of open models, but having something dangled in front of you but suddenly unable to verify is not a good feeling.
With the right amount of coaxing you can sometimes get it to reproduce a PoC with a "TDD first" approach to fixing, but often now that gets blocked too.
So far I can always get around it by starting fresh sessions on a similar problem and without ever mentioning security issues, but why do I have to play this little theater with a tool we pay to use.
On the other hand this sounds like a job guarantee, though. And not only for the moment, as it is not the capability of the model which is insufficient.
I'm not opposed to open models at all, I'm just slightly sceptical on how much some people push their use compared to the value we get out of a dirt cheap subscription.
Propriety models are clearly subsidised by enterprise API contracts, so it makes sense to milk that for all its worth, provided one steers clear of any lock in so the option to switch is always there.
Jumping to spending time managing and self-hosting open models is a nice ideal but I think it's a distraction to jump on that train without proven benefits given the value that other plans deliver.
I don't mean I'm against the idea of open models, they were especially instrumental in Anthropic backing down from the "Fable only initially available 2 weeks" plan, which got extended up until Fable was over-taken by Opus in general usefulness.
It's my view that any company sized appropriately for Teams plan should be milking it for all its worth while it still exists, and it is a waste to lose that opportunity by jumping on open models too early.
If you're a larger enterprise then it would be a very different story, you'd already be paying more forced onto API pricing and would likely have the capacity to have a team look at open models without impacting delivery.
there is a middle ground between proprietary models and self-hosting: Fireworks, et al. Popular models like DeepSeek have a couple dozen providers on openrouter.
There is, but it also has an additional trust layer on top, where you have to trust you're getting back responses from the models you're being sold and aren't being artificially quantised or stealthily redirected to cheaper models.
I trust ( although some don't ), Anthropic not to be doing that. I don't trust intermediate providers so much. If you're self-hosting then at least you know exactly what is running and get privacy on top.
Likewise you lose privacy of local hosting, and if it emerged that a provider was training on data they shouldn't be, would you have the necessary legal recourse, that you would going direct to the major model providers?
If only Claude didn't constantly drop to less capable models whenever it encountered anything vaguely cybersecurity related (despite being part of the Anthropic Cyber Verification Program and having Mythos access, albeit limited).
We're strongly pushing for a move to a mix of Codex and open models at my workplace because of Anthropic's ridiculous nanny-ing.
These types of flip flop posts are the new security theater. One week Anthropic will post a doom and gloom about how open weight models running a fraction of a token per second on your Grandma's popped Android TV is a national security threat and the next week they'll tell you how awesome their 200+ billion parameter model is at doing something they don't actually allow you to do because they gatekeep all the normies. I really do hope history remembers them correctly, but I'm sure they'll be attributed as being a significant and load bearing linchpin during the new world order of LLMs.
Also in the CVP and kept tripping cyber a couple days ago due to an MCP server that had the ability to connect to remote devices. Maybe it thought I was trying to make a C&C server.
Yep, gotta really mind the language, sometimes saying something like “when implementing <foo> feature you gotta pay attention to <bar> to avoid <baz> attack” is enough to trigger it, along with a cache miss of 300k tokens in the downgraded request to rub it in. Frustrating.
Having a big brother on the frontier models in the cyber verification program defeats the whole purpose of the program. My team has found ourselves leaning the same direction your team is going. I was looking into GLM 5.3 to host in our env and come to find there is an article from anthropic trying to say that model is "dangerous" and/or not good compared to their models.
There is really just one way this goes and its upwards to the peak: which means full formal verification programming languages, OS uni-kernels, small TCB and probably verified HW for the most sensitive cases. Everything else will soon be hackable via 'brute compute' because economy of scale and accessibility of open weight models.
These programs are executed in environments. Which are run on a hardware. Big part of which is hidden software.
And in general you can have 5 boxes verified and be surprised what comes out of them calling each other. Number of boxes is growing and possible interactions is exp. I don't think formal verification solves much.
I would argue: if each each part (box) of a system is truly formally verified, it would only expose a set interface (API) that is also formally verified by definition, and all possible interactions are also of a closed set and formally verified.
How hard such closed sets are (or even possible) of course varies by domain and application.
I'm doing this attack surface tactic with Vix IDE, a terminal text editor that helps novices get comfortable with vi, emacs, git, jj, sql, commands, and more.
The prompt I used was "Goal: rework to reduce the attack surface. Process: move features into feature-flag Rust crates. Output: build a minimal version. Questions: Ask me anything."
This past weekend, Claude Fable/Opus/Sonnet together successfully refactored the whole Vix IDE codebase into many separate Rust crates and Cargo features.
This refactor enables a security-oriented user to pick and choose, and also makes it easier for a security auditor to look at one capability at a time.
If you're curious you can see the commit messages by Claude Sonnet. IMHO they're reasonably good.
Is there not a great irony in over a weekend throwing Fable/Opus/Sonnet at your project, refactoring your whole codebase, and the purpose of that project is to teach people to use vi, git, sql and so on. Why would anyone want to learn that in the current day and age, when as you state it can refactor a project over a weekend fairly autonomously?
Ha! Yes you're right. IMHO the industry is in a limited window of time where software engineers are still using the foundational tooling and not quite yet using AI for autonomous creating. This is especially true in regulated sectors.
This sounds so arrogant, lol. I know it actually means that they red team the frontier AI models (which is a reasonable claim), but it looks like they are claiming to be the best red team in the world
I would believe Anthropic about how Claude reduces attack surfaces if they weren't simultaneously nannying everyone about how they supposedly don't want their models used for cybersecurity because... Reasons. And if you do want that access, you have to verify your identity, and do a bunch of other things, all of which goes against "reducing" the "attack surface". If they really gave a damn about cybersecurity, this cybersecurity filter wouldn't exist, period.
I’m very frustrated by this, for the record. But they have the atom bomb, and we want to borrow their tech for nuclear power. They don’t trust us. This familiar geopolitical quagmire isn’t resolved on the global scale, so I don’t expect these flawed companies to do it any better.
One can phrase it: bad actors already have nukes, and throwing them on peaceful American cities every day, and gov + corps deny citizen access to defensive tools while do nothing for protecting citizens on their side too.
Recently I watched this presentation of a kernel developer about the "Mythos' 79 kernel vulnerabilities": https://youtu.be/NnV_cWeoo5Q?t=257
He claims that most of these vulnerabilities are not an issue at all, or have been fixed earlier, etc. I can clearly see the improvement of the AI models, but based on this youtube video it's very plausible that these AI labs rush to announce a result without further understanding of the underlying problems. I wonder what's the quality here in this article.
That's the weird thing with LLMs, they know just enough about a topic to fool anyone without much knowledge on the subject, seemingly even frontier labs.
It doesn't help that the labs have positioned themselves on the predication that expertise is no longer required.
They are trying to sell this false myth of machines so good they can replace any human expert, and they buy this BS themselves, so, instead of actually learning something and/or confirming results with institutions of experts (what's left of them anyway) they just rush out a PR piece knowing the unwitting public and VCs won't care enough to look into the validity of anything themselves anyway.
The whole thing is a gamble on whether or not they can convince the world that a fancy word machine substitutes for deep experience and expertise.
(To be clear, the fancy word machines are still very useful tools, but they are not adequate replacements for knowing humans).
Yes, and before that you were probably adept at writing code based on incomplete and sometimes inaccurate documentation, using tools and technologies that were powerful but occasionally buggy.
Or at the very least, you were interested in the subjects in question and just never had the time or other resources to learn them.
Yeah the rate at which I'm learning is insane! I think the key is picking a project with practical goals that you can track towards, ideally backed with concrete evidence (like data) and measuring progress by how close you are getting to those goals via working code.
As such, I'm learning through motivated, self-directed, focused exploration, using the AI to answer questions, run experiments, explain findings, investigate surprises, and explore new directions to research, all grounded in concrete data related to those goals.
The data and our results on it are what keep us honest. The data is also what helps understanding nuances and building intuition because the AI can explain outcomes in the context of samples data points that you can reason about independently.
The difference between YOLO vibe-coding and engineering is understanding. I think this is the future of learning and skill-building.
I don't get what your point is WRT this article. They aren't announcing a result? They're describing a technique they used in real projects to prevent new vulnerabilities, including a change that is merged into OpenSSL. What is the "rush to announce a result" here? What are you saying they don't understand?
There is an unresolved issue where the offensive guys can use AI to attack a target and the defensive guys are being limited when they ask AI to audit their codebase.
The workflow for a defender basically goes:
Claude do a security audit on this codebase, use the harness / skills I built on how to do security audits.. pick the best suitable audit strategy, methods.. use the tooling and reporting structures.
Claude refuses, GPT refuses.
Grok, goes further, but refuses to make a working PoC.. a requirement because AI models make stuff up and without a PoC there is no proof.
GLM-5.3 / Kimi to the rescue and completes the job the "smarter" models refused.
So we as defenders are refused by the best models, on our own codebase
Heck, Codex was trying to use gdb to debug a capture crash and the session locked up with security blocks. I had to restore a checkpoint and tell it abandon that work and stay away from gdb.
I've heard people getting jammed up with Claude refusing to update DNS..
There's nothing stopping you from applying for Daybreak for OpenAI / Unsure which program Anthropic has. But yes, you do have to share your identity and I can see how that is a blocker for some.
Why are offensive users not subject to the same limitations? Assuming it is because they are using different models, why aren't the defenders doing the same?
So rich to have them post this when just this morning I had to switch to GLM 5.3 so I could mitigate the PHP CVEs (both Claude and Codex balked at the early investigation work). Once I was further along I was able to switch back to Claude to implement the fix but it refused to look at any POC's or test them against my dev servers, so freaking annoying.
The only things worse than the CVE getting dropped in our laps with little warning was having Claude run into its safeguards which slowed my progress in actually protecting our customers.
Last week Claude (and Codex) also refused to help write unit tests around existing code because it touched on authentication. I was able to ladder between them past everywhere they got stuck but today I went straight to GLM after hitting safeguards.
Anthropic keeps talking about using its models to detect security issues in existing projects and other people's code, but no mention of its models creating security issues in new code and the mass of vulnerabilities AI agents are creating in its wake. If you run Fable on Opus code, how many security issues do you find? The irony is Fable can detect many of them, but its security guardrails refuses to fix them.
60 comments
[ 0.25 ms ] story [ 51.3 ms ] threadI'm not that hot on the idea of open models, but having something dangled in front of you but suddenly unable to verify is not a good feeling.
With the right amount of coaxing you can sometimes get it to reproduce a PoC with a "TDD first" approach to fixing, but often now that gets blocked too.
Propriety models are clearly subsidised by enterprise API contracts, so it makes sense to milk that for all its worth, provided one steers clear of any lock in so the option to switch is always there.
Jumping to spending time managing and self-hosting open models is a nice ideal but I think it's a distraction to jump on that train without proven benefits given the value that other plans deliver.
I don't mean I'm against the idea of open models, they were especially instrumental in Anthropic backing down from the "Fable only initially available 2 weeks" plan, which got extended up until Fable was over-taken by Opus in general usefulness.
It's my view that any company sized appropriately for Teams plan should be milking it for all its worth while it still exists, and it is a waste to lose that opportunity by jumping on open models too early.
If you're a larger enterprise then it would be a very different story, you'd already be paying more forced onto API pricing and would likely have the capacity to have a team look at open models without impacting delivery.
I trust ( although some don't ), Anthropic not to be doing that. I don't trust intermediate providers so much. If you're self-hosting then at least you know exactly what is running and get privacy on top.
Likewise you lose privacy of local hosting, and if it emerged that a provider was training on data they shouldn't be, would you have the necessary legal recourse, that you would going direct to the major model providers?
I think the hackers are just using "not Claude".
We're strongly pushing for a move to a mix of Codex and open models at my workplace because of Anthropic's ridiculous nanny-ing.
It's a 10T BTW.
And in general you can have 5 boxes verified and be surprised what comes out of them calling each other. Number of boxes is growing and possible interactions is exp. I don't think formal verification solves much.
How hard such closed sets are (or even possible) of course varies by domain and application.
The prompt I used was "Goal: rework to reduce the attack surface. Process: move features into feature-flag Rust crates. Output: build a minimal version. Questions: Ask me anything."
https://github.com/vixide/vix
This past weekend, Claude Fable/Opus/Sonnet together successfully refactored the whole Vix IDE codebase into many separate Rust crates and Cargo features.
This refactor enables a security-oriented user to pick and choose, and also makes it easier for a security auditor to look at one capability at a time.
If you're curious you can see the commit messages by Claude Sonnet. IMHO they're reasonably good.
This sounds so arrogant, lol. I know it actually means that they red team the frontier AI models (which is a reasonable claim), but it looks like they are claiming to be the best red team in the world
Missing an apostrophe? A human wrote this! Love to see it.
He claims that most of these vulnerabilities are not an issue at all, or have been fixed earlier, etc. I can clearly see the improvement of the AI models, but based on this youtube video it's very plausible that these AI labs rush to announce a result without further understanding of the underlying problems. I wonder what's the quality here in this article.
They are trying to sell this false myth of machines so good they can replace any human expert, and they buy this BS themselves, so, instead of actually learning something and/or confirming results with institutions of experts (what's left of them anyway) they just rush out a PR piece knowing the unwitting public and VCs won't care enough to look into the validity of anything themselves anyway.
The whole thing is a gamble on whether or not they can convince the world that a fancy word machine substitutes for deep experience and expertise.
(To be clear, the fancy word machines are still very useful tools, but they are not adequate replacements for knowing humans).
There are humans like this and they are some of the most catastrophic people to have on a project, I actively avoid them.
I'm sure LLMs are different.
How do I know I didn't learn hallucinations?
Because based on that knowledge, I've produced code that works as intended.
Or at the very least, you were interested in the subjects in question and just never had the time or other resources to learn them.
As such, I'm learning through motivated, self-directed, focused exploration, using the AI to answer questions, run experiments, explain findings, investigate surprises, and explore new directions to research, all grounded in concrete data related to those goals.
The data and our results on it are what keep us honest. The data is also what helps understanding nuances and building intuition because the AI can explain outcomes in the context of samples data points that you can reason about independently.
The difference between YOLO vibe-coding and engineering is understanding. I think this is the future of learning and skill-building.
God, I'd hate to be on the receiving end of their bug/issue submissions.
The workflow for a defender basically goes: Claude do a security audit on this codebase, use the harness / skills I built on how to do security audits.. pick the best suitable audit strategy, methods.. use the tooling and reporting structures.
Claude refuses, GPT refuses. Grok, goes further, but refuses to make a working PoC.. a requirement because AI models make stuff up and without a PoC there is no proof. GLM-5.3 / Kimi to the rescue and completes the job the "smarter" models refused.
So we as defenders are refused by the best models, on our own codebase
All projects need security and writing code means taking security decisions, you can't untangle both.
I've heard people getting jammed up with Claude refusing to update DNS..
The situation is pretty insane right now.
The only things worse than the CVE getting dropped in our laps with little warning was having Claude run into its safeguards which slowed my progress in actually protecting our customers.
Last week Claude (and Codex) also refused to help write unit tests around existing code because it touched on authentication. I was able to ladder between them past everywhere they got stuck but today I went straight to GLM after hitting safeguards.