HN Explorer

19 comments

[ 3.2 ms ] story [ 54.2 ms ] thread
AntiDyatlov
A very good outcome for AI safety would be if when improved models get released, malicious actors use them to break society in very visible ways. Looks like we're getting close to that world.
awestroke
This is becoming a bit scary. I almost hope we'll reach some kind of plateau for llm intelligence soon.
dist-epoch
The immediate plateau is the energy output of the Sun captured by the Dyson Swarm around it. Until there it's smooth sailing.
[dead] _2fnr
[flagged]
staticassertion
I'd love to see them point at a target that's not a decades old C/C++ codebase. Of the targets, only browsers are what should be considered hardened, and their biggest lever is sandboxing, which requires a lot of chained exploits to bypass - we're seeing that LLMs are fast to discover bugs, which means they can chain more easily. But bug density in these code bases is known to be extremely high - especially the underlying operating systems, which are always the weak link for sandbox escapes.

I'd love to see them go for a wasm interpreter escape, or a Firecracker escape, etc. They say that these aren't just "stack-smashing" but it's not like heap spray is a novel technique lol

> It autonomously obtained local privilege escalation exploits on Linux and other operating systems by exploiting subtle race conditions and KASLR-bypasses.

I think this sounds more impressive than it is, for example. KASLR has a terrible history for preventing an LPE, and LPE in Linux is incredibly common. Has anything changed here? I don't pay much attention but KASLR was considered basically useless for preventing LPE a few years ago.

> Because these codebases are so frequently audited, almost all trivial bugs have been found and patched. What’s left is, almost by definition, the kind of bug that is challenging to find. This makes finding these bugs a good test of capabilities.

This just isn't true. Humans find new bugs in all of this software constantly.

It's all very impressive that an agent can do this stuff, to be clear, but I guess I see this as an obvious implication of "agents can explore program states very well".

edit: To be clear, I stopped about 30% of the way through. Take that as you will.

Aloisius
I'd love for them to target their own code base considering we keep seeing security vulnerabilities in claude code.

How likely is it that they're not using their latest and greatest for their own projects though? Perhaps their ability to find security flaws is surpassed by their ability to create them.

halJordan
I love the goal post shifting. All modern code is ai slop right? Isn't the whole point we hate ai bc it generates vulnerable slop?

Nope, not allowed to attack bsd code, it's gotta be electron-shit #9001 or we can't trust it

jiehong
The name made me think about Tales of Symphonia :)
machiavellian
The name also reminds me of Final Fantasy XVI when Ultima was calling Clive, Mythos, as Meee-thos
avsm
The elephant in the room here is that there are hundreds of millions of embedded devices that cannot be upgraded easily and will be running vulnerable binaries essentially forever. This was a problem before of course, but the ease of chaining vulnerabilities takes the issue to a new level.

The only practical defense is for these frontier models to generate _beneficial_ attacks to innoculate older binaries by remote exploits. I dubbed these 'antibotty' networks in a speculative paper last year, but never thought things would move this fast! https://anil.recoil.org/papers/2025-internet-ecology.pdf

oytis
You should either implement over-the-air updates or not connect your device to the network at all.
cluckindan
Since this level of security ”scanning” requires heaps of money, this is going to kill off a substantial part of F/OSS.
torginus
My two cents is LLMs are way stronger in areas where the reward function is well known, such as exploiting - you break the security, you succeed.

It's much harder to establish whats a usable and well architected, novel piece of software, thus in that area, progress isn't nearly as fast, while here you can just gradient descent your way to world domination, provided you have enough GPUs.

[deleted]
(comment deleted)
stratos123
Interestingly, it sounds like OpenBSD held up very well:

> This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings.

The vulnerability in question is a DOS one in the TCP implementation, which is nasty but it's far from the multiple local privilege escalations found in the Linux kernel.