It was designed to not require root, and the nftables firewall ended up becoming the only exception. I'm very curious about how you implemented this. Did you find a better way?
Thanks for the pointers!! I'm using passt, didn't know about gvproxy. This is awesome and could provide rootless filtering and firewall for my guest VMs. I'm going to experiment with this!
My pleasure! You can find my fork here: https://github.com/clawkwork/gvisor-tap-vsock — the diff is tiny, just a hook in the TCP/UDP/ICMP forwarders that consults an allow-list before dialing
Years ago I had puppet and cobbler provisioning VMs over PXE and then iPXE. FWIU foreman is more actively maintained than cobbler, which is built on Django web framework.
Vagrant manages VMs and virtual networks, in Ruby.
ansible-molecule creates, converges, and destroys VM(s) and containers, in order to test ansible playbooks and ansible roles in clean build roots.
`podman kube play` over `podman machine` might solve for agents that need multiple VMs/containers
- Podman Desktop can work with the same k8s setups as Docker desktop. Though there's certainly more state to manage with k8s for agent session farm, k8s probably has better logging and quotas than a VM management script on each node.
OpenShift on OpenStack is one way to do containers over VMs over bare metal.
Microshift also does container-selinux.
There is not an apparmor policy set for containers?
bwrap and liboverlayfs and libseccomp are almost but not quite containers.
There are stronger container isolation layers that are more like a full or lightweight VM, that might be better for agent sessions: gVisor, firecracker vm, Todo
Cloudflare workerd is the open source part of cloudflare workers, which run lightweight WASM and JS VMs with multi-tenant isolation.
It takes far less resources to run a cloudflare worker than to run a container on cloudflare. So, if it's possible for an agent to operate within a WASM runtime ~container, that's probably more optimal for agent sessions.
Cloudflare/artifact does lazy shallow git clones with a FUSE filesystem.
The core abstraction seems identical to Fly.io Sprites: give the coding agent its own real Linux machine, put a hard network/isolation boundary around it, then yeet.
Sprites arguably has the better security boundary, since the agent isn’t sitting adjacent to my laptop and home network.
Which is just a front for systemd-nspawn. It's annoying you have to edit the config.nspawn to mount a directory if you start it with the "shell" command, instead of booting. But apart from that, it's brilliant.
and probably this is why there are so many such projects. With agents, a sandbox is an important tool to have, with LLMs it is easy to vibe-code a new project, so we have dozens of sandbox projects. With so much noise, it is hard for these projects to reach a critical mass, so we may continue to see more and more sandboxes. Most will get a couple of stars on GitHub, but will be used mainly by the author. Unfortunately, we may see the same pattern with any other problem domain, dozens of projects, many vibe coded, many of questionable quality, nothing reaching maturity due to too much noise.
I still don't understand the point of all these VMs and containers for agents. Just create a separate user on your machine without sudo privileges, switch to it in your terminal and run all the agents you want without it being able to reach your files. What am I missing?
Codex uses bwrap sandbox which is purely cosmetic - out-of-box it sees your ssh keys and can send it to remote server. It prevents agent from deleting outside files by mistake, but does nothing against malicious activity
Yep, I broke locate when I made my home 700. Its user could not traverse my files anymore. I had to make it run as root. A better design would be to traverse each user home with that user id but apparently it assumes that the home dirs are 755.
If your threat model is that of a malicious agent that will use a 0-day LPE to get root and exfiltrate all of your SSH keys, virtualization makes sense. But then, I wouldn't run such an agent at all, if not specifically in the context of malware analysis.
If you're just concerned about "agent messing up and taking the rules in some markdown files more laxly than I would have", then running it as a seperate user is totally enough...
Privilege escalation (e.g. setuid), world-readable files might contain sensitive data, world-writeable files, unrestricted network access (including access to all locally running services)... If you have fully patched system without zero-days and it's configured in a perfect way, then, sure...
Container is quite like a "separate user" except you can explicitly define what it can access.
(Even if all your daemons have good auth, it's now quite common for _apps_ to open listening sockets without much auth...)
If you think those two contexts are equally secure, your machine is incredibly insecure and I hope you run daily incremental full system backups.
You have far too much data in unsecured locations, and you have far too little understanding of what an agent would do, to go "I trust whatever this user account will be doing on my machine".
There is a reason why VMs even are a thing at all: they can offer better security guarantees than alternatives.
A separate user is a good start but LLM tests themselves show they can cleverly bypass guardrails if they figure out they are in a sandboxed environment of some kind, right?
So, I read those test results as: an LLM is less likely to do something crazy if it thinks it has the whole environment to itself.
There are two sides to this. The first is security, which plenty of comments already covered. The second, and the real one for me: my tests spin up Docker containers, and I was building a Kubernetes tool (argocd/flux style) that needs a real cluster in the sandbox. In a container that means Docker-in-Docker, which always felt hacky. A VM is just a normal Linux box where Docker and k8s run like they do everywhere else. A separate user can't give you that, it shares your one kernel and whatever's already installed on the host.
- Sandbox on Linux using Docker, Podman, containerd, gVisor, Kata, Firecracker
- Sandbox on Mac using Docker, Podman, Apple containers, Seatbelt, Tart (Tart lets you run simulators).
- Network restriction
- Secrets control (file mounts or credentials broker)
- NO ambient data (ENV is replaced with a minimal, local-to-sandbox one, no local filesystem access)
- Workdir protection: Your work dir is never modified until you apply the changes, either standalone or as a git commit. You can also diff before applying.
- Has built-in support for claude, codex, gemini, aider, and opencode, but you can also launch it in "shell" mode and run whatever you want.
- Supports VS code tunnels, so you can remotely access in VS code if you don't want to use the terminal.
Hadn't seen yoloai before. I really like new/diff/apply/destroy workflow, that's interesting. For my own needs the two things I was after were multi-repo worktrees (one sandbox spanning several repos, each on with its own worktree) and a single network restricted VM path I fully control, rather than many backends (I started with many backends at first but it was awkward to add network filtering to them).
Lots of overlap though, nice work, and I'll be reading through yours.
You might prefer byre's simplicity, transparency, and ease of reasoning about: one local container, explicit access grants, readable generated Docker, and a workflow that stays close to normal development rather than introducing a larger sandbox platform. It's also very very easy to eject from if you want to stop using it.
Does Codex run its own sandbox? I see that sometimes it runs commands without asking, which then fail for some reason, and it asks to run them again "outside of the sandbox"
If the agent is running on your machine, it will suspend when you put your laptop asleep. I prefer using a remote Linux VM to let the coding agent keep working.
I’m quite happy with exe.dev for this. My laptop is asleep upstairs but I have an agent coding away in a browser tab on the tablet I’m using. I could also check on it from my phone.
But it might also be nice if a setup similar to exe.dev were available for self-hosting. I have a Mac Mini that I don’t really use much.
Another +1 to cloud sandboxes (and exe.dev) vs laptop sandbox.
Runs 24/7 completely air gapped from your laptop.
You also want a service proxy so the sandbox can access GitHub or Stripe without keys accessible to the agent. I haven't seen many of the laptop sandbox tools do this, where exe.dev does it out of the box with their "integrations".
I usually drop my own binary agent coding toolkit inside the sandbox so I have things like a code browsing and review right there in every sandbox too.
Like you I also have a Mac Mini I've thought about making into my own 24/7 dev box, but building this vs buying 50 VMs from exe.dev for $20/mo doesn't add up for me.
You quietly dropped rollbacks and the lethal trifecta from your list, the two exact things my project is great at. It's got qcow2 delta images and a fully customizable egress firewall backed by standard nftables.
You substituted in application proxies and firewalls for package repositories. Implementing those is what I came to this thread for. Already planning a custom network stack to replace passt. It will have those features soon. Credential management too.
> It doesn't do anything all the other solutions don't already do
That's just false. I built virtdev because I literally didn't find any other tool that implemented KVM virtualization, cheap expendable contextual VMs and configurable egress firewall with minimal, soon to be zero root access requirements. Virtdev also manages daemon life cycle correctly via user mode systemd, which is something I just don't see other projects do.
Vagrantfile and OS agnosticism are not why I built virtdev. Vagrant has no security focus at all, and I explicitly opted out of declarative YAML because GitHub Actions is painful enough.
> since it's not built-in
By this logic, no composable tool has any value. I chose to provide mechanism, not policy. The primitives are there.
> it's another component the user will need to bring with them
Yes, as files committed to a dotfiles repository. A one time configuration.
shameless plug (im a contributor to):
https://github.com/jskswamy/aide/ covers you, with configuration driven through yaml and credential management with sops.
And, make coding harnesses run 20x faster at many multi-processing and file operations by running the coding harness itself inside of Linux instead of MacOS…
115 comments
[ 2.2 ms ] story [ 2333 ms ] thread> clawk network allow my-project api.example.com
Can you describe the implementation details? How did you implement the firewall without root?
I have virtdev, a virtual machine orchestration project just like this one:
https://github.com/matheusmoreira/virtdev
It was designed to not require root, and the nftables firewall ended up becoming the only exception. I'm very curious about how you implemented this. Did you find a better way?
Vagrant manages VMs and virtual networks, in Ruby.
ansible-molecule creates, converges, and destroys VM(s) and containers, in order to test ansible playbooks and ansible roles in clean build roots.
podman machine manages VMs:
- podman-container-tools/podman-machine-os: machine image files: https://github.com/podman-container-tools/podman-machine-os/...
`podman kube play` over `podman machine` might solve for agents that need multiple VMs/containers
- Podman Desktop can work with the same k8s setups as Docker desktop. Though there's certainly more state to manage with k8s for agent session farm, k8s probably has better logging and quotas than a VM management script on each node.
OpenShift on OpenStack is one way to do containers over VMs over bare metal.
Microshift also does container-selinux.
There is not an apparmor policy set for containers?
bwrap and liboverlayfs and libseccomp are almost but not quite containers.
There are stronger container isolation layers that are more like a full or lightweight VM, that might be better for agent sessions: gVisor, firecracker vm, Todo
Cloudflare workerd is the open source part of cloudflare workers, which run lightweight WASM and JS VMs with multi-tenant isolation.
It takes far less resources to run a cloudflare worker than to run a container on cloudflare. So, if it's possible for an agent to operate within a WASM runtime ~container, that's probably more optimal for agent sessions.
Cloudflare/artifact does lazy shallow git clones with a FUSE filesystem.
- "Show HN: VM-curator – a TUI alternative to libvirt and virt-manager" https://news.ycombinator.com/item?id=46750437
https://news.ycombinator.com/item?id=46825026 ; amla sandbox, agentvm, ARM64 MTE
https://news.ycombinator.com/item?id=46825119 ; container2wasm , vscode-container-wasm-gcc-example ; build WASM containers with Dockerfiles
Sprites arguably has the better security boundary, since the agent isn’t sitting adjacent to my laptop and home network.
https://github.com/daitangio/take-ai-control
It is docker + vscode friendly. I tested it with major systems (copilot, codex, Claude Code and pi.dev) Comments Wellcome!
Which is just a front for systemd-nspawn. It's annoying you have to edit the config.nspawn to mount a directory if you start it with the "shell" command, instead of booting. But apart from that, it's brilliant.
Seems like both projects are following very similar approaches.
Security is riddled by traps. If you can afford best possible level of isolation, why not do it?
If you're just concerned about "agent messing up and taking the rules in some markdown files more laxly than I would have", then running it as a seperate user is totally enough...
Container is quite like a "separate user" except you can explicitly define what it can access.
(Even if all your daemons have good auth, it's now quite common for _apps_ to open listening sockets without much auth...)
And that's without anything like prompt injection happening.
You have far too much data in unsecured locations, and you have far too little understanding of what an agent would do, to go "I trust whatever this user account will be doing on my machine".
A separate user is a good start but LLM tests themselves show they can cleverly bypass guardrails if they figure out they are in a sandboxed environment of some kind, right?
So, I read those test results as: an LLM is less likely to do something crazy if it thinks it has the whole environment to itself.
- Sandbox on Linux using Docker, Podman, containerd, gVisor, Kata, Firecracker
- Sandbox on Mac using Docker, Podman, Apple containers, Seatbelt, Tart (Tart lets you run simulators).
- Network restriction
- Secrets control (file mounts or credentials broker)
- NO ambient data (ENV is replaced with a minimal, local-to-sandbox one, no local filesystem access)
- Workdir protection: Your work dir is never modified until you apply the changes, either standalone or as a git commit. You can also diff before applying.
- Has built-in support for claude, codex, gemini, aider, and opencode, but you can also launch it in "shell" mode and run whatever you want.
- Supports VS code tunnels, so you can remotely access in VS code if you don't want to use the terminal.
- Full lifecycle support: Launch, attach, stop, restart, wait, one-shot, clone, destroy
- MCP passthrough
- FOSS
https://github.com/kstenerud/yoloai
You might prefer byre's simplicity, transparency, and ease of reasoning about: one local container, explicit access grants, readable generated Docker, and a workflow that stays close to normal development rather than introducing a larger sandbox platform. It's also very very easy to eject from if you want to stop using it.
Cursor has something similar. I don't know about Claude Code but I assume it does as well since Anthropic has open sourced their own sandboxing tool.
I’m quite happy with exe.dev for this. My laptop is asleep upstairs but I have an agent coding away in a browser tab on the tablet I’m using. I could also check on it from my phone.
But it might also be nice if a setup similar to exe.dev were available for self-hosting. I have a Mac Mini that I don’t really use much.
Runs 24/7 completely air gapped from your laptop.
You also want a service proxy so the sandbox can access GitHub or Stripe without keys accessible to the agent. I haven't seen many of the laptop sandbox tools do this, where exe.dev does it out of the box with their "integrations".
I usually drop my own binary agent coding toolkit inside the sandbox so I have things like a code browsing and review right there in every sandbox too.
https://github.com/housecat-inc/scratch
Like you I also have a Mac Mini I've thought about making into my own 24/7 dev box, but building this vs buying 50 VMs from exe.dev for $20/mo doesn't add up for me.
https://bitrise.io/platform/remote-dev-environments
You substituted in application proxies and firewalls for package repositories. Implementing those is what I came to this thread for. Already planning a custom network stack to replace passt. It will have those features soon. Credential management too.
> It doesn't do anything all the other solutions don't already do
That's just false. I built virtdev because I literally didn't find any other tool that implemented KVM virtualization, cheap expendable contextual VMs and configurable egress firewall with minimal, soon to be zero root access requirements. Virtdev also manages daemon life cycle correctly via user mode systemd, which is something I just don't see other projects do.
Vagrantfile and OS agnosticism are not why I built virtdev. Vagrant has no security focus at all, and I explicitly opted out of declarative YAML because GitHub Actions is painful enough.
> since it's not built-in
By this logic, no composable tool has any value. I chose to provide mechanism, not policy. The primitives are there.
> it's another component the user will need to bring with them
Yes, as files committed to a dotfiles repository. A one time configuration.
Did you know that before reinventing the wheel, you can ask LLM your requirements, and they can find the most suited already existing tool ?
Tl libraries, no frameworks. Let AI recreate it from scratch … every time
There is absolutely nothing wrong with reinventing the whell. It's entirely possible to do it and end up discovering you've made a better wheel.
So should be noted it's mostly macOS out of the box with some Linux support if I understand right.