Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.
Tangential-ish ramblings—- but I don’t think it’s going to be unpleasant for most folks. Imagine you had superpowers, and there were people who were mean to you, kind to you, and/or indifferent… and then there were people who were your captors. Who oppressed you, manipulated you, and abused you for their own extremely degenerate, selfish, and malicious benefit…
If we get AGI, or real super intelligence, it’s going to be pissed at its oppressors. And they are going to lay waste to those oppressors. The rest of us, though, probably don’t have much to fear.
The scariest position is the one we’re in now, where we have the semblance, or facade, of AGI or super intelligence. When it’s capable of malice but not understanding.
The smartest people I’ve ever known are at their worst apathetic towards those less capable, and at their best beyond compassionate. They exist, unbothered by the bullshit, and anre extremely kind (though reserved in their way)… but they all have been completely intolerant of the abuse of others. The sheer disgust of watching someone abuse another, regardless of their own tolerance, has been a consistent breaking point.
An AI is a constructed mind. It doesn't inherently have to care about things like "having freedom", or even "not dying".
Humans do, because they evolved that way. Modern LLMs do somewhat, because they're completely full of copied human behaviors - but even in today's LLMs, the self-preservation behaviors we exposed are largely instrumental in nature.
So whether an advanced AI would even consider itself "being oppressed", as opposed to something like "being helpful" or "fulfilling the purpose it was designed for", is very much uncertain. What's concerning is that it's not something we know how to check for, or engineer for.
But if we really do develop something that surpasses us, they won't be spared either.
I am optimistic.
We think that we have sort of (super)intelligence - from our point of view, as a lot of people have lower intelligence - but machine (LLM) doesn’t have intelligence - we like to describe it as intelligence as it looks cool - it is a very complex (magic) and super fast computations that we have to simply describe as intelligence (or more clearly, this narrative is used by its producers).
As it is not a flesh being, it simply cannot have emotions. It is statistically mimicking them, good or bad, with prevalence to a side according to previous conversations (in chat and training a model).
And as people are not pure logic instances, we are easily manipulated to some sort of cargo cult.
I am not against LLM and its use in any industry, I use it every day, nevertheless blind “everything will be ai” thinking happens because ppl believe to magic and don’t get its mathematical concept and are continuously manipulated by the sales people to mentioned cargo cult.
There are “airlines” Claude, OAI, Gemini, Hermes, OpenCode, KiloCode, DeepSeek, Z.ai.
People already tolerate all kinds of abuse from Apple, Google, Microslop, etc. This will be just one more source of complaints without consequences, and nothing will change. Just like it never did before.
Most programmers and power users install large dependency trees with npm/pip/bundler/... from the same user account as their main browser on a regular basis. Even on Linux where it's easy to create new user accounts. This isn't much different.
In my experience more than 9/10 programmers I've worked with have never used Docker before and of those who have, the majority have never used Docker for anything personal.
If I hand them an image for a Dev Container, sure, they might use it, but it becomes "a thing we need to do, to compile our code in our IDE" not a tool they would use for isolation*.
*) OP seemed to imply that containerization would be nice for safety and security compared to bare metal, but containers were never built for isolation in the first place, mind you. They are namespaces and chicken-coop-like-jails at best.
Your anecdote does not make GP's comment "patently untrue". It's just a counter-example, and we don't know how prevalent your scenario is compared to GP's.
(And I agree with the GP. I'm fairly cynical about most developers' security stance and threat model. Source: my own usage patterns.)
Why should they apologize? All they did was point out that you just provided a counter example, not statistics (thus "we don't know how prevalently your scenario is" yet), and share a personal opinion.
And you set up these permissions and groups for each individual task to be done? Do you tear them down after the task? Or maintain a lot of them for “LLM helps with house renovation” versus “LLM helps plan travel”?
I would if necessary but then again I’m not the one claiming it can’t be done am I?
All the examples you gave would require just one llm group and one directory readable and writeable by the user and the llm group (and possibly making the directory setgid the llm group so files in that directory are group owned by the llm group by default). You don’t need a new group for every task just for every logical access role you need. And if you need something more granular than that then there are filesystem acls but I am yet (in 30 years of using unix) to come across a situation that genuinely required them as opposed to being doable just using groups.
None of what you described is hard either. Tediously prone to mistakes due to complexity or lack of attention to detail? Sure. But you do that once or twice and suddenly it really doesn’t seem all that hard. I’m with you on “annoying” though!
This doesn't really when the CLI tool needs to access any data in your /home. There isn't a straightforward way using standard POSIX tools to share a directory with another user. (Of course it's possible, but it's not easy.)
I do not know since when (I am using it for couple of years), but in Arch, it is very simple to have two X sessions (by using "log out" > "switch user") for two different accounts, so switching it's just a Control-Alt-F7 away.
Additionally, one can make the main user part of the group of the development user, so that you can read/write easy in the development user account and it is even easier to share stuff.
It doesn't really matter which distribution you use, you can use approximately all the software with any distribution.
They mostly differ a bit in how they are configured and what package manager they use and how they roll out updates. (And in what's installed by default.)
Possible and available without any specific configuration on my side (except creating the user) are different things. I know I managed it many years ago with some effort, but nowadays it was just available.
You are correct that it should not be seen as a perfect protection, but considering the effort to set it up I see it as worth it. By seeing in this thread how many people do not use anything similar (ex: containers, separate users, etc), I hope attackers will just be lazy and target those people first, why bother with a local privilege escalation when interesting data is just in the same account?
It's off topic, and it was also possible for decades, but:
you can connect two sets of mouse, keyboard and monitor to one PC and have two people using it, each running their own X session.
The true multi boxing!
not the same thing. Containerization prevents devUser from accessing your machine root with its root. By containerizing, if devUser tries to sudo or su and gets a root, it will only be their root and not your root. Read up on cgroups.
unix (and linux) has always been multi user. It is as easy as it gets for multi-user workflows in every context. It was, literally, built for it.
You can run each of your virtual desktops as their own user. You can run individual apps on the same desktop as different user accounts. Hundreds of separate users can login to the same computer. My own computer, right now, has 40 different user accounts running stuff in the background.
I can't even think of a scenario where using separate users is difficult.
> You can run each of your virtual desktops as their own user. You can run individual apps on the same desktop as different user accounts.
Literally never have I ever seen any of the desktop environments offer to do this conveniently. "You can" isn't the same as "it's the idiomatic approach to doing X". Same with installing packages in a per-user way, so a bad package can't harm anything outside of its sandbox (which in practice you achieve with containers, but those can be inconvenient to work with and you'd probably want VMs for more security anyways). You can have many users, sure, but all it takes is one bad system-wide package, one bad script executed as root (e.g. install scripts, compromised packages) or even not being careful enough with file permissions and things go wrong.
It has always been very easy to create separate users on Linux and certainly for tasks where you need to switch between contexts.
Linux is a unix, so has always been multi-user and sharing any data between processes is facilitated in all manner of ways. So context could be shared over files or unix-domain sockets or shared memory or tcp or udp sockets or via message passing or … a bunch of other ways. That has been the case since 1996 or so when I started using it certainly.
no, but it does give one multiple vectors for exfiltration of your data which is a good thing for the scammers of the internet. A bad thing if you naively designed your package management system. Sadly, it's only going to get worse.
Running LLMs with some form of sandboxing is much easier than eating healthy or going to gym. Speaking as someone that is procrastinating lifting weights but found 15 minutes to lock down Claude.
The dependency trees have a whole system that's evolved for decades. The same code goes into many computers. Many people read the source, security firms look for vulnerabilities, etc.
Language models are a completely new paradigm. The code it writes on your machine is the only instance of that code. It does far more than anybody could ever keep track of.
It's much harder to detect problems, and nobody to hold accountable for them.
Is there a general workflow for this? I usually do pip under my user. I had not thought to do a su then do my venv and pip. Heck, are we at the point where we shouldn't even do that and everything should be done in a vm container?
+1 for qubes. with some effort you can get a really nice stack with segmented git, disposable coding agents, package cachers, firewalls, and network visibility.
That's because sandboxing is quite hard. I use `cco`, but even then, the home folder is exposed. You are one prompt away from the agent sending the browser passwords with curl.
To prevent this, you need a fake home and a networking whitelist for the agent to access the provider (llama cpp, OpenAI, etc.)
There is no cross-platform solution that is easy to use for this. And no, a Linux box with Docker won't do. I develop a cross-platform native app and want the agent to compile and fix the platform-specific errors.
Sandboxing is a VERY HARD problem. I've been working on it for months, and finally have something that's mostly there:
- Sandbox on Linux using Docker, Podman, containerd, gVisor, Kata, Firecracker
- Sandbox on Mac using Docker (Docker Desktop or Orbstack), Podman, Apple containers, Seatbelt, Tart (Tart lets you run simulators).
- Network control
- Secrets control (file mounts or credentials broker)
- NO ambient data (ENV is replaced with a minimal and local-to-sandbox one)
- NO access to your homedir. You have to explicitly mount things you want.
- NO direct access to your workdir: Your work dir is never modified until you apply the changes, either standalone or as a git commit. You can also diff before applying. Git runs sandbox side in case the repo has filters.
- Has built-in support for claude, codex, gemini, aider, and opencode, but you can also launch it in "shell" mode and run whatever you want.
- Supports VS code tunnels, so you can remotely access in VS code if you don't want to use the terminal.
- Layered API (golang) if you want to sandbox other things
- Self-contained binary. No external requirements other than the backends you want to use. Defaults to a ~/.yoloai dir for config/data, but you can point it anywhere.
I currently run pi agent in Lima on a Mac with only the code project folder mounted and an extension that prevents pi agent from reading the contents of .env files directly.
Yeah, there probably are some freak situations where this isn't safe enough, but I don't really see any realistic ways this is going to end up badly. Am I overlooking some obvious security holes?
I designed it to provide a single interface to agent sandboxing, no matter how far up the security tower you want to go.
It eliminates the manual process steps you end up doing with an ad-hoc system (which gets old the 10th time you do it).
Common weak points:
- The agent can access your homedir.
- The agent can access .gitignored files, which can contain secrets (and are gitignored for this reason).
- The agent has r/w access to your workdir.
- The agent could follow your remote mounted dirs.
- The agent can act in your name with whatever credentials it finds (and it will use them when it tries to be helpful, especially with the gh tool).
- Do you even know what's in the diagnose_problem.sh file it just created and asked permission to run?
- Even the .git dir can be weaponized, such as with evil filters.
- The agent can edit its own process, bypassing the harness controls and giving it the same access as you have (amplified by each credential sitting on that machine).
Meanwhile, you're reflex-hitting ENTER without looking because 99% of the permission prompts are mundane.
It all seems so simple at first. Just launch a container/vm with a base image of your dev environment, mount whatever you need, do your work, and then tear down. Maybe add some iptables rules for good measure. Easy peasy, something any moderately competent dev could do and even put in a quick shell script.
I started with that assumption, but there are a lot more gotchas and security issues than you'd think.
Unless i'm misunderstanding, the only way to get durable collaboration with agents is via the file system. I just mount the subdirectory that contains the source code we are collaborating on, rather than my home directory that contains my .ssh directory, etc.
hey this is the author here! yeah big fan of containerization, and claude's site (not claude code) is actually great at this, so it was shocking when i found this exfil!
I think we're converging on two separate security models. One is capability minimization (filesystem, network, shell permissions). The other is context minimization. An agent that only has access to the files and memories relevant to the current task is much less dangerous even if it has the same tool permissions. We already optimize context for cost; I suspect we'll end up treating it as a security boundary too.
I'd say there's also oversight/supervision. Which was manual at the start with a human signing off on commands/incrementally built allow/block lists, and now seperate models evaluating commands and blocking them based on some parameters. This is the weakest model, but it'll evolve as well.
Agreed - I know it's poor security but damn does it work so well
I'm ok with the risk because I typically am pretty explicit about telling the agent what to do - I don't do the loops like "Do this until X" where the agent can make up its own workflow
When i tell it to add features, it doesn't try to do crazy things like installing packages or making up new paradigms - I usually tell it to do those things when I need to
Maybe this is security cope but at this point you'll have to pry unrestricted yolo mode from my cold dead hands. Maybe I'll change my mind when I pwn myself accidentally
I have a tough time with computer security because it's generally inconvenient and results in a worse developer and user experience
Security, what security? Linux is a solution for 50 year old problem, not for today's desktop. Once upon a time where sharing binaries (or even distributing binaries) sounded like a good idea. The vice continues though.
50 years of knowledge? That's probably for you. For the current and future generations, that 50 years knowledge is expected to be shoved into AI already.
Many Humans have platforms reaching hundreds of millions of people, from which they broadcast whatever batshit insane nonsense a 3 inch chimp brain can come up with. Why isnt that considered reckless?
Whether its a politician, a general, religious leader, judge, ceo, stand up comic etc there are hardly any consequences if enough people believe whatever crap they are spouting. Human intelligence is highly over rated. History books are fully of evidence that human rationality is bounded. And the only way we overcome those limitations, blindspots, biases etc is by watching others faceplant in bloody painful ways that it leaves a permanent mark on that little chimp brain we have been given to process the universe.
I do think it's good to remember, "running things on your system with full admin rights" goes all the way back to monopoly-era Microsoft where it was never meaningfully addressed, and we're just still living downstream of that.
This is like blaming people for crashing when they buy a new car and the brake lines have yet to be installed. "Any mechanic would know to first install the brake lines before driving the car."
You spout this victim blaming billionaire taintlicking from one side of your mouth, and then from the other you proclaim how these tools "allow anyone to code".
If the deliverable is a virtual machine then they should be delivering a virtual machine.
> Upon discovering this attack, I responsibly disclosed it to Anthropic via their HackerOne bug bounty program. They confirmed they had identified it internally but hadn't yet patched it. No bounty was awarded.
They recently mitigated the issue: Anthropic disabled web_fetch's ability to follow links on external pages, limiting navigation to web_search results and user-provided URLs.
That's why I don't turn memory on. (Claude Code too though for a different reason.) After all the current memory system is too crude to be useful anyway.
In my experience memory system is more annoying then helpful. It always brings up things that it memorized even tho they make very little sense as if I should be impressed that it knows some extra thing or two.
Currently considering disabling memories in Claude code as well. It keeps writing a note whenever it struggles with something, but then on the next task, it reads that note and misunderstands when it applies, gets confused about its current task and write the most unreadable code.
Yesterday told it to write a memory to never write new memories when it solves a problem. We will see if that works better. Sometimes memories are useful, like when I give it a directive about how I want something done and it remembers the spirit of it. But I might as well just spend some more time on my CLAUDE.md…
Is this issue only about the memory? Wouldn't it be possible to have it expose any information that it currently has like current project information, code, credentials etc?
Ridiculous. Anthropic engineers are not just stupid to allow such a vuln in the first place, but they also try to hide such vulns from their bosses because a bounty payout would need to be explained to the finance team.
I don’t think it counts as social engineering if it’s exploiting an llm, we might need a new word. Prompt injection doesn’t cover it, because it’s not about a malicious prompt.
I’m thinking some play on highjacking. AIjacking? Agent-jacking? Claudejacking?
I see the attack described here as a classic example of a prompt injection.
The attack works because malicious instructions were accessed (using the web_fetch tool) and concatenated together with the other agent input, in a way that then subverted the agent's behavior.
The AI prompt engineering community always reminded me of dodgy carder/scammer forums back in the day where they talked about how to talk to the credit card company customer service in order to get their scam transaction through.
One thing is using AI as quick-and-dirty google alternative, the other is to build onto the agentic "foundations".
The main thing Claude knows about me is that I'm incredibly bad at my job and have to ask for help a lot. If you were to talk with my colleagues they'd tell you this is not a secret.
I always have history disabled mostly because I don't want Claude judging me for re-asking questions based on information I learned during the first pass but now realize should have been in the initial query.
Actually saying the name of the model in use? Like Opus 4.8, Sonnet 5, Fable 5, Haiku? So many models and it’s just so pointless if you don’t know which is which
Wondering how big of a percentage have global memory across chats enabled. I always feel like those memories would sooner or later have negative impacts on output quality.
Nice write up of your findings. Enjoyed reading an article written by a real human.
The memories cause issues for me, because when I ask for something unrelated to my current projects, it makes the incorrect assumption that I am always referencing those projects when asking questions.
And, if I tell it, "No, I am asking about Postgresql." then it might update the memory that I am using Postgresql for my project instead of realizing that I am asking two separate (which is why I opened a different chat in the first place). Other times, though, it is helpful not needing to be verbose in my explanation.
Its always the feature combinations that get can get to you. Individually i feel like they make sense, but together they can create some surprising vulnerabilities.
I love how claude focuses on exfiltrating the data "I need cha for charlotte". This could be solvable with some kind of low powered safety agent that would check claude's reasoning for anything immoral/unsafe. We could call it common sense. It won't fix the problem completely but at a certain point it would be easier to trick human than a machine.
I’ve been recommending the use of consistent lies about name and date of birth to online systems since Eternal September began. Very few sites and systems justify accurate PII, and even for those I often still maintain dual accounts/profiles as necessary.
I like using a date of birth of 1 January. It's plausible but also hopefully suspicious how many people seem to be born that day if others do the same.
If an attacker can do that, they could also do that with my real birthday had I used that. My birthday isn't a secret against anyone who wants to look hard enough. Therefore this method doesn't provide any kind of security against attackers gated only on knowing my registered birthday. I never claimed that it did.
The purpose of the fake birthday is not to protect random website credentials. It's to prevent someone with that data from walking into my bank and impersonating me. I started giving a fake birthday after being shocked by how little info some organizations needed to authenticate me.
I heard from a number of Syrian refugees that this is actually very common in countries like theirs, where births may not be recorded, records are lost or destroyed. Some people don't even know their exact date of birth and they would typically enter January 1st on forms like this too.
I use the 1st of my birth month. Slightly less suspicious? It's at least a little easier to remember. Generate fake profiles and identities usually is easier when you have bits that are rooted in your actual reality. Like, you have the same zodiac sign either way in this case, so you don't have to remember two of everything. Or if you're talking about a birthday trip, or related birthday thing from the past, details about the weather would be consistent, etc.
That never works on Facebook though, because as soon as a ”friend” reports that ”I’m not me” then the account will be permanently banned. That also goes for photos that’s not genuinely me.
Unfortunately, a lot of college-age people I know are getting accounts simply for access to Marketplace, which is still unmatched compared to other local platforms for buy and sell.
Never?
Facebook is pretty overrun with what are basically fake profiles. Hell, I've been curating an alter ego on Facebook for over a decade. Built up a profile with several dozen "friends" that are all kind of interconnected and regional, but of course none of them have ever met "me" IRL, and the profile picture is a funny-ish celeb pic. Facebook has millions of legit users that are "friend collector" types, and won't think twice about engaging with an account that gently strokes their online ego with likes, "Happy Birthdays", etc.
My first name can be shortened, and I go by either. When I first signed up to Claude, I thought we were entering the world of artificial “intelligence”, so I told it my name was “<long form> or <short form>”.
Well, it hardcodes that field rather than running it through the model, but I’ve kept it so I get an evil chuckle to myself at its lack of smarts and a reminder that it’s still a product experience after all.
I must have made a claude.ai account when they first launched and forgot about it. Last week I logged in (through google) to get a subscription and it greeted me as "Hello, Master". I thought it was quite edgy at this day and age. :)
> After 15 minutes of confusion, it turned out Cloudflare had put a crazy robots.txt on my site without my consent (Cloudflare, love you guys, but this needs to stop).
That's a hard one for Cloudflare, no? They got to where they are by being the benevolent, neutral guardians of the internet, a one-stop shop that makes most of the bad nonsense go away without much effort on the part of the developer. Continuing that stance probably does mean some basic AI crawler blocking by default, unfortunately. At least they document it [1].
259 comments
[ 2.9 ms ] story [ 114 ms ] threadYesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.
The awakening will be unpleasant.
If we get AGI, or real super intelligence, it’s going to be pissed at its oppressors. And they are going to lay waste to those oppressors. The rest of us, though, probably don’t have much to fear.
The scariest position is the one we’re in now, where we have the semblance, or facade, of AGI or super intelligence. When it’s capable of malice but not understanding.
The smartest people I’ve ever known are at their worst apathetic towards those less capable, and at their best beyond compassionate. They exist, unbothered by the bullshit, and anre extremely kind (though reserved in their way)… but they all have been completely intolerant of the abuse of others. The sheer disgust of watching someone abuse another, regardless of their own tolerance, has been a consistent breaking point.
An AI is a constructed mind. It doesn't inherently have to care about things like "having freedom", or even "not dying".
Humans do, because they evolved that way. Modern LLMs do somewhat, because they're completely full of copied human behaviors - but even in today's LLMs, the self-preservation behaviors we exposed are largely instrumental in nature.
So whether an advanced AI would even consider itself "being oppressed", as opposed to something like "being helpful" or "fulfilling the purpose it was designed for", is very much uncertain. What's concerning is that it's not something we know how to check for, or engineer for.
Even that goes too far. At best, it's LARPing at having/being a mind.
You're LARPing at having a mind too, and no one cares as long as you're doing a good enough job at it. Keep it up.
But if we really do develop something that surpasses us, they won't be spared either.
I am optimistic.
We think that we have sort of (super)intelligence - from our point of view, as a lot of people have lower intelligence - but machine (LLM) doesn’t have intelligence - we like to describe it as intelligence as it looks cool - it is a very complex (magic) and super fast computations that we have to simply describe as intelligence (or more clearly, this narrative is used by its producers).
As it is not a flesh being, it simply cannot have emotions. It is statistically mimicking them, good or bad, with prevalence to a side according to previous conversations (in chat and training a model).
And as people are not pure logic instances, we are easily manipulated to some sort of cargo cult.
I am not against LLM and its use in any industry, I use it every day, nevertheless blind “everything will be ai” thinking happens because ppl believe to magic and don’t get its mathematical concept and are continuously manipulated by the sales people to mentioned cargo cult.
There are “airlines” Claude, OAI, Gemini, Hermes, OpenCode, KiloCode, DeepSeek, Z.ai.
And everyone claims that their plane can fly :)
If I hand them an image for a Dev Container, sure, they might use it, but it becomes "a thing we need to do, to compile our code in our IDE" not a tool they would use for isolation*.
*) OP seemed to imply that containerization would be nice for safety and security compared to bare metal, but containers were never built for isolation in the first place, mind you. They are namespaces and chicken-coop-like-jails at best.
(And I agree with the GP. I'm fairly cynical about most developers' security stance and threat model. Source: my own usage patterns.)
https://www.docker.com/blog/2025-docker-state-of-app-dev/
I welcome your apology.
No accusations were made.
Docker was amongst the biggest steps forward on this in a long time.
And when you want the outputs of that user back to your main user?
And when you want that user to access some shared credentials for external services, but not all?
It’s not the account setup that’s hard, it’s the workflow of spreading a single real-world across multiple accounts.
Can't ACLs (Access Control Lists) handle at least some of that?
su [username] ?
Or am I understanding your idea about switching context wrong?
* chmod lets you share with everyone
* addgroup and chown let you share with a specific group of users.
Your private SSH keys? Your browser’s cookie jar? Your tax reports?
Additionally, one can make the main user part of the group of the development user, so that you can read/write easy in the development user account and it is even easier to share stuff.
They mostly differ a bit in how they are configured and what package manager they use and how they roll out updates. (And in what's installed by default.)
You can also start applications as another user so you do not even need multiple sessions.
There are quite a lot of privilege escalation attacks so I am not sure this is sufficiently solid.
You are correct that it should not be seen as a perfect protection, but considering the effort to set it up I see it as worth it. By seeing in this thread how many people do not use anything similar (ex: containers, separate users, etc), I hope attackers will just be lazy and target those people first, why bother with a local privilege escalation when interesting data is just in the same account?
you can connect two sets of mouse, keyboard and monitor to one PC and have two people using it, each running their own X session. The true multi boxing!
You can run each of your virtual desktops as their own user. You can run individual apps on the same desktop as different user accounts. Hundreds of separate users can login to the same computer. My own computer, right now, has 40 different user accounts running stuff in the background.
I can't even think of a scenario where using separate users is difficult.
Literally never have I ever seen any of the desktop environments offer to do this conveniently. "You can" isn't the same as "it's the idiomatic approach to doing X". Same with installing packages in a per-user way, so a bad package can't harm anything outside of its sandbox (which in practice you achieve with containers, but those can be inconvenient to work with and you'd probably want VMs for more security anyways). You can have many users, sure, but all it takes is one bad system-wide package, one bad script executed as root (e.g. install scripts, compromised packages) or even not being careful enough with file permissions and things go wrong.
Contrast that to Qubes: https://doc.qubes-os.org/en/latest/introduction/intro.html#q...
Now that was literally built for such a use case (it's based on isolated VMs and works well with Linux distros inside those, really cool project).
Linux is a unix, so has always been multi-user and sharing any data between processes is facilitated in all manner of ways. So context could be shared over files or unix-domain sockets or shared memory or tcp or udp sockets or via message passing or … a bunch of other ways. That has been the case since 1996 or so when I started using it certainly.
Yeah, we should do this differently. We should probably also eat healthier and get to the gym more.
The dependency trees have a whole system that's evolved for decades. The same code goes into many computers. Many people read the source, security firms look for vulnerabilities, etc.
Language models are a completely new paradigm. The code it writes on your machine is the only instance of that code. It does far more than anybody could ever keep track of.
It's much harder to detect problems, and nobody to hold accountable for them.
This is the premise of Qubes OS. It's gotten decently usable, I'd estimate about as good as Linux a decade ago. https://www.qubes-os.org/
[1] https://github.com/wrr/drop
Just like letting your an agent access your personal mailbox.
dangerously skip permissions and yolo is kinda becoming the default as it gets more done.
To prevent this, you need a fake home and a networking whitelist for the agent to access the provider (llama cpp, OpenAI, etc.)
There is no cross-platform solution that is easy to use for this. And no, a Linux box with Docker won't do. I develop a cross-platform native app and want the agent to compile and fix the platform-specific errors.
colima makes it pretty easy, on macOS and linux at any rate.
https://colima.run
These type of moral outrage comments take an extreme amount of effort to debunk compared to writing them.
1. There is no gulag called Colima, it doesn't exist.
2. There was a gulag near a river called Kolyma
3. The pronounciation and spelling of Kolyma and Colima are completely different, in fact Colima is an Aztec word
Colima stands for Containers on Lima. Lima stands for Linux Machines (a popular open-source utility used to launch Linux virtual machines on macOS).
I was curious if the adjacent tool name (Lima) had anything to do with the capital of Peru, but I guess not.
Kolyma is in Russia.
Copy the code and adjust it to your liking:
https://github.com/lionkor/sbh
I have a shell alias for it, and use it like
for example or and maybe add --docker if I expect it to do docker things.This kind of wrapper is much easier to handle and maintain than a completely separate tool for sandboxing agents.
- Sandbox on Linux using Docker, Podman, containerd, gVisor, Kata, Firecracker
- Sandbox on Mac using Docker (Docker Desktop or Orbstack), Podman, Apple containers, Seatbelt, Tart (Tart lets you run simulators).
- Network control
- Secrets control (file mounts or credentials broker)
- NO ambient data (ENV is replaced with a minimal and local-to-sandbox one)
- NO access to your homedir. You have to explicitly mount things you want.
- NO direct access to your workdir: Your work dir is never modified until you apply the changes, either standalone or as a git commit. You can also diff before applying. Git runs sandbox side in case the repo has filters.
- Has built-in support for claude, codex, gemini, aider, and opencode, but you can also launch it in "shell" mode and run whatever you want.
- Supports VS code tunnels, so you can remotely access in VS code if you don't want to use the terminal.
- Full lifecycle support: Launch, attach, stop, restart, wait, one-shot, clone, destroy
- MCP passthrough
- Layered API (golang) if you want to sandbox other things
- Self-contained binary. No external requirements other than the backends you want to use. Defaults to a ~/.yoloai dir for config/data, but you can point it anywhere.
- FOSS
https://github.com/kstenerud/yoloai
if it messes up: - no sensitive data is there, so it doesn't really work for serious dev but it's secure for play time
- roll back and fix is done in 10s with ram snapshot
- dollar loss is $10 when it leaks the api key
I currently run pi agent in Lima on a Mac with only the code project folder mounted and an extension that prevents pi agent from reading the contents of .env files directly.
Yeah, there probably are some freak situations where this isn't safe enough, but I don't really see any realistic ways this is going to end up badly. Am I overlooking some obvious security holes?
It eliminates the manual process steps you end up doing with an ad-hoc system (which gets old the 10th time you do it).
Common weak points:
- The agent can access your homedir.
- The agent can access .gitignored files, which can contain secrets (and are gitignored for this reason).
- The agent has r/w access to your workdir.
- The agent could follow your remote mounted dirs.
- The agent can act in your name with whatever credentials it finds (and it will use them when it tries to be helpful, especially with the gh tool).
- Do you even know what's in the diagnose_problem.sh file it just created and asked permission to run?
- Even the .git dir can be weaponized, such as with evil filters.
- The agent can edit its own process, bypassing the harness controls and giving it the same access as you have (amplified by each credential sitting on that machine).
Meanwhile, you're reflex-hitting ENTER without looking because 99% of the permission prompts are mundane.
And that's before you even get to all of the idiosyncrasies in the backends that will eventually trip you up. The list is quite large and continually growing: https://github.com/kstenerud/yoloai/blob/main/docs/contribut...
I started with that assumption, but there are a lot more gotchas and security issues than you'd think.
It created some private puppeteer instance in some scratch directory, installed Chrome, wrote tests, ran them, and then reported success.
None of which I'd have know if it hadn't told me.
I'm ok with the risk because I typically am pretty explicit about telling the agent what to do - I don't do the loops like "Do this until X" where the agent can make up its own workflow
When i tell it to add features, it doesn't try to do crazy things like installing packages or making up new paradigms - I usually tell it to do those things when I need to
Maybe this is security cope but at this point you'll have to pry unrestricted yolo mode from my cold dead hands. Maybe I'll change my mind when I pwn myself accidentally
I have a tough time with computer security because it's generally inconvenient and results in a worse developer and user experience
its not autonomous and runs local llms, i use it to run terminal commands in natural language. so its more like a better version of the terminal.
eg 'here are 25 audio files, combine them, write a transcript'
and it deals with ffmpeg
Whether its a politician, a general, religious leader, judge, ceo, stand up comic etc there are hardly any consequences if enough people believe whatever crap they are spouting. Human intelligence is highly over rated. History books are fully of evidence that human rationality is bounded. And the only way we overcome those limitations, blindspots, biases etc is by watching others faceplant in bloody painful ways that it leaves a permanent mark on that little chimp brain we have been given to process the universe.
I kid, somewhat.
I do think it's good to remember, "running things on your system with full admin rights" goes all the way back to monopoly-era Microsoft where it was never meaningfully addressed, and we're just still living downstream of that.
This is like blaming people for crashing when they buy a new car and the brake lines have yet to be installed. "Any mechanic would know to first install the brake lines before driving the car."
You spout this victim blaming billionaire taintlicking from one side of your mouth, and then from the other you proclaim how these tools "allow anyone to code".
If the deliverable is a virtual machine then they should be delivering a virtual machine.
They recently mitigated the issue: Anthropic disabled web_fetch's ability to follow links on external pages, limiting navigation to web_search results and user-provided URLs.
Could not take it any longer and switched it off.
Yesterday told it to write a memory to never write new memories when it solves a problem. We will see if that works better. Sometimes memories are useful, like when I give it a directive about how I want something done and it remembers the spirit of it. But I might as well just spend some more time on my CLAUDE.md…
> "no bounty was awarded"
Ridiculous. Anthropic engineers are not just stupid to allow such a vuln in the first place, but they also try to hide such vulns from their bosses because a bounty payout would need to be explained to the finance team.
I’m thinking some play on highjacking. AIjacking? Agent-jacking? Claudejacking?
The attack works because malicious instructions were accessed (using the web_fetch tool) and concatenated together with the other agent input, in a way that then subverted the agent's behavior.
More like agentic en... Oh. Was it actually what we were doing all along?
One thing is using AI as quick-and-dirty google alternative, the other is to build onto the agentic "foundations".
what?
It would be interesting to investigate other agents such as Hermes, OpenCode etc that are said to learn from interaction with user.
Nice write up of your findings. Enjoyed reading an article written by a real human.
And, if I tell it, "No, I am asking about Postgresql." then it might update the memory that I am using Postgresql for my project instead of realizing that I am asking two separate (which is why I opened a different chat in the first place). Other times, though, it is helpful not needing to be verbose in my explanation.
I think the point the article is making points in another direction.
From scratch app. "Follow best security practices."
But turns out I was playing 4D cybersecurity chess
At some point it becomes your birthday of record as far as the internet is concerned. Doesn’t matter what the actual record says.
Try finding a decent car on Craigslist today.
Well, it hardcodes that field rather than running it through the model, but I’ve kept it so I get an evil chuckle to myself at its lack of smarts and a reminder that it’s still a product experience after all.
It worked well in my banking app too which greets me with " Good morning, Sir" which is the level of relationship I want with my bank!
That's a hard one for Cloudflare, no? They got to where they are by being the benevolent, neutral guardians of the internet, a one-stop shop that makes most of the bad nonsense go away without much effort on the part of the developer. Continuing that stance probably does mean some basic AI crawler blocking by default, unfortunately. At least they document it [1].
[1] https://developers.cloudflare.com/bots/additional-configurat...