systemd is not the only thing using linux namespaces. For example, the suggested bubblewrap uses namespaces to allow isolated networks (--unshare-net) and changing rootfs (--bind /mnt/foo /)
One of the fake recruiters contacted me on LikedIN for a Solidity dev role, with an assignment repo that came as a zip in DM. Everything looked fine since Finder only showed .sol files, but Finder doesn't show .git and other hidden folders by default. However .git was there, and it had a hook that ran eval() on a remote address on commit/push.
As I read it, .git protection stops the VM from changing hooks, and --clone leaves the hook behind since a clone doesn't copy hooks. However, the hook is still in the folder on the host. Does devvm inspect and warn when a folder already has hooks before you start?
I see nothing this brings over smolvm machines. I did not use lima / krunkit outside colima but in colima it was just slow, had outdated devx and was unreliable compared to controlling every detail with an embedded vm library.
This is something that could run on top of smolvm, if the authors chose it, rather than Lima/QEMU. It both sets up virtual machines and what goes into them, letting you chooose toolchains and coding agents. smolvm doesn't provide dev tool setup within a vm, though it does point to images that already have toolchains and agents installed.
Indeed, but this new niche project is trying to provide a streamlined setup. Upon closer inspection of the toolchains, for Node, Rust, Go, and Python, it shows that it tries to automatically configure them.
This sort of wrapping setup of VMs in a higher level project is encouraged by BinSquare, the creator of smol machines, as recently on X he has been touting its embeddability.
I have built my agent harness around smolvm and it has been working wonderfully.
I have seen a single bug up to now (iirc you cannot take a snapshot from a machine you rehydrate from a pack) but found a workaround.
I think the only thing I would love a solution for is the ability to bind credentials even when you use tailscale. You cannot right now because the traffic goes through the tailscale interface.
This looks quite similar to docker sandbox (sbx). It uses Lima for VMs under the hood. Windows advertised as not supported (Lima is missing features on Windows).
Like Docker Sandbox this meets what I think should be our standards in the AI age: kernel isolation and secret shielding in addition to network and disk isolation.
The things that are design choices for these tools are
* agent integration
* docker image composition
* are vms more persistent or throw away
* managing configuration
I have been working on a similar project because I haven't seen one before that to me didn't have design deficiencies. This project looks good to me but I need to try it out now and experience the design choices.
I missed the one directory part, I'd definitely recommend making that more flexible. With my equivalent of this I often mount multiple repositories into a single VM, two examples:
- a bunch of gitops repos: terraform mono repo, argocd-esk conf repo, grafana provisioning repo, etc.
- frontend + backend repo (tbh may merge these to a mono-repo, but here we are)
I think the auto detection is interesting, but tbh with my solution I've been happy with just install all the tools regardless of what a specific project needs - with a COW base layer this is cheap.
It's probably a fine project, but the instructions are like "trust this thing, give us your login" , which still gets alarm bells ringing in my head pretty fast.
Heh, yeah also very similar. I don't strip devices like you are - I actually want the memory balloon functionality for example.
My working threat model for my projects is less about "malicious agent trys to escape VM" and more "well meaning agent got confused and screwed up" - the added protection of a VM is desirable, and it's useful for stuff like nested virtualization (agent working on improving the VM orchestration), makes containers inside the sandbox straightforward
I hadn't seen the stuff about filtering spoofed packets so I'll probably add that.
One thing I'm struggling with at the moment which I may as well ask here.
Many of my projects use pnpm, which ideally uses hard links from a shared store of dependencies.
One of my goals is to reduce the disk overhead of VM based isolation, so I'd like a shared pnpm store across projects (and the same for maven, etc as well)
With https://virtio-fs.gitlab.io/ mounts, each mount inside the VM is a separate FS, and I can't hard link across them. I've tried playing around with ideas like bind mounting the directory structure from the host in a folder to have a single virtio-fs mount inside the VM - but the bind mounts still can't be hard linked across it seems.
I'm basically wondering if I'm missing something or just trying to do something that isn't possible.
All the host folders are on the same FS - but I don't want to organise the host system around the VM mounting needs, and I also don't want to mount extraneous directories into the VM (eg: simple solution could be just mount ~/projects but kinda self defeating)
Add me to the list, although I use podman containers (https://github.com/evertheylen/probox). I find that the hard part is not restricting the container/VM, but to make it "easy" to do things in the container that require something protected. How will you commit? Where does your agent run, and can it access github? How will you deploy? How will you SSH to a server?
Happy to hear what other people are thinking. It seems like putting an intercepting HTTP proxy is getting popular in other projects.
On the git access part, I don't let my agent VMs have write access to GitHub currently. I do give them dedicated users on a self hosted gitea instance, and basically manage the blast radius through branch protection etc.
I think at a high level I'm trying to treat them as junior/intern employees where I give them enough permission to do their role, but not enough rope to hang themselves.
I see. But then I have to debug some failing CI pipeline and the agent is quite good at going through the logs, so I give in and `gh auth login` anyway...
I'm pretty sure you don't need to give it full access to accomplish that. It's possible to create scoped PAT etc. alternatively use a separate account. Ultimately your risk tolerance to decide
You can have a separate GitHub account for your agent (GitHub ToS actually allows that since April) and that account can fork your repo and open PRs. This way it cannot do any real damage but can still push.
Do you have more background on this? My team actually ran into some issues with GH when we tried this earlier this year (felt like it was around that same time).
We have a few simple rules for Accounts on GitHub's Service.
You must be a human to create an Account. Accounts registered by "bots" or other automated methods are not permitted. We do permit machine accounts:
A machine account is an Account set up by an individual human who accepts the Terms on behalf of the Account, provides a valid email address, and is responsible for its actions. A machine account is used exclusively for performing automated tasks. Multiple users may direct the actions of a machine account, but the owner of the Account is ultimately responsible for the machine's actions. You may maintain no more than one free machine account in addition to your free Personal Account.
One person or legal entity may maintain no more than one free Account (if you choose to control a machine account as well, that's fine, but it can only be used for running a machine).
I usually commit outside the sandbox. Agents will see the git history and will emulate that but they can't sign the commits. If I authorize a series of commits I can just `git rebase -i HEAD~N` and adjust the messages and re-sign at the same time. With git hooks as a threat vector I'm considering just giving them read-only access to `.git` otherwise I can't safely go back in there without the sandbox.
I don't give it access to GH - it can read most things via API or web scraping fast enough and I manually run the commands I need outside the sandbox.
No need for SSH - it's running locally.
---
Given how many holes you're poking into the container I think you'd be better served with a real VM.
add me as well. I learned about incus from HN ~a month ago, and was poking an idea of having a "development base image".
The advantage compared to containers, is that you can hot-plug devices to the running instance: mounts, secrets, etc.
incus also supports --vm and container mode with minimal difference in cli commands.
my main bottleneck was exactly in usability: how to make it least effort to spin up per use case and keep the set of operating scripts small - i didn't want to build an app for this, just shortcuts to use existing app.
Also been working on something similar, with qemu/kvm; the idea is to sandbox agents and limit resource usage per project. Its not production-ready yet, but the repo is public - https://github.com/blackshieldpt/catchpen
You're not wrong about the curl install method, but expecting projects to get into distro repositories just because they want to is unreasonable and if they ran their own little package repo or just distributed packages from more manual install, many of the same dangers and other problems with curl-to-shell methods could still be present.
I just made my own as an excuse to practice my podman skills. It's saved me countless times over and over. It doesn't have git installed at all and doesn't even mount .git so I feel safe nothing can be FUBAR'd.
I tried to develop an electron app this way; ubuntu VM (using qemu/multipass) on macos as host with Intellij IDEA. Couldnt make it work properly. Sure, I can install all the toolchain in the VM, and Intellij would recognize it (they have a “remote dev” setup), but for instance for electron apps you cannot just run a headless vm, and there is no convinient way to make a change in your mac, hot reload in the VM, and interact with the electron app in your mac.
I keep seeing projects like this pop up and they never nail it.
I need isolation, but I also need to give my agents secure access to internal package registries, logs, internal infra, designs, emulators, browsers, docs etc. At a large company, work can span dozens of repositories. And on top of that, I need dozens of these.
I built this at work. Engineers use krunkit VMs with proper memory ballooning, GPU acceleration. Retina RDP, credential injecting proxy for etc.
This seems like a fun toy for single project development, but nothing close to what you need as an engineer in a corporate environment.
I think this sort of thing can still be useful to smaller shops. I have a hobby project in this vein, and I know it will never make it big, but it's still a fun hamster wheel.
Because it's not just VMs, but a lot more around credentials injection, egress, volumes, agents selection policies. I am quite sure also the memory ballooning and GPU acceleration can be fine tuned.
In advance, thank you very much for your feedback!
You got the right base, but it seems centered around single project/ticket work.
A lot of work me and my fellow engineers do spans weeks if not months, tens of code bases, tickets etc. Having longer lived sandboxes that cover N repos in which you can just let experiments exist, together with plans etc is a superpower.
It’s an option on the Windows app (Microsoft’s own RDP client), connects at full retina resolution but while having the host being set to 200% UI scale, so that you get the crisp text and UI.
I'm constantly blown away by the power that LLM's are giving us. I've been working on a self-improving agent for around six months - and over the past few weeks I used VM's to isolate the governance layer. Virtualization is - to me - a completely black art, and yet with the deepseek flash 4.1 model the agent itself has built it's own virtual machine launcher/manager on OSX and even managed to port it to windows (windows Home has very limited VM support). For all the negative excitement about LLM's, I'm just amazed how our capabilities are being extended as engineers and what's possible.
I made something like this at work as a way to spin up multiple dev stacks on the same project. It’s docker based though so not the best isolation.
The problem i having is more workflow related. AI is doing the dev so fast I constantly need to validate different versions and change sets of an app. Git worktrees work for isolating the agents changes but doesn’t solve running multiple instances of a dev server and remembering which one applies to which agent session.
The solution I came up with was throw the code and all tools in a container git branch. Then run a reverse proxy to get <mybranch>.localhost style urls working.
Would be interested to give this a shot if it can also solve for some of those workflow issues.
I'm looking for a solution that lets me experiment with random projects in isolated environments (could be Docker containers or VMs), seamlessly, but I want to run VSCode on my actual machine, and whatever agent extensions.
I tried the remote SSH thing and it's painful to setup. Did anyone find a better solution than scripting a "create Docker container, bind the gpu, bindmount an external directory for data, install code server, etc."?
"You know you should run your code and agents in a sandbox, but most tools need configuration and setup that’s hard to get right."
This is very true. Having been a user of heavily-customised vagrant boxes for a long time I've been spending a fair bit of time lately on this.
But my own stupid old-fashioned view is that it's probably worth learning cloud-init if you never have. Deploy your VMs on your own machine with multipass, same deployment works with little change in the cloud.
That's been the approach I have settled on, but only after a disappointing experience with Apple's Container Machine, which is both quite bothersomely insecure by default (your whole home directory? mounted rw? in a VM with an agent? really?) and from there sort of difficult to configure more effectively, and weirdly idiosyncratic where DNS/networking is concerned.
Because SRT is just wrapper around Seatbelt(macOS)/Bubblewrap(Linux), with some additional stuff. The process runs on the host kernel, which makes it vulnerable to kernel exploits. And you still have to write the config file properly, which is quite annoying in some situations.
While running the Agent inside a VM/MicroVM means you're mainly exposed through potential hypervisor exploits, and potentially to vulnerabilities in your custom Host-Guest interface. Which makes it safer, once you've set it up, it covers a broader set of use case out of the box, unlike srt where you might have to keep changing your config file.
That's my experience after trying SRT and a few MicroVM setups though. I still use both, just for different stuff, but random code always goes straight into the MicroVM.
76 comments
[ 0.26 ms ] story [ 11.8 ms ] thread`sbx`, safehouse and the like
I'm unsure how to differentiate between all these projects
Bubblewrap would just be a better alternative for this.
As I read it, .git protection stops the VM from changing hooks, and --clone leaves the hook behind since a clone doesn't copy hooks. However, the hook is still in the folder on the host. Does devvm inspect and warn when a folder already has hooks before you start?
This sort of wrapping setup of VMs in a higher level project is encouraged by BinSquare, the creator of smol machines, as recently on X he has been touting its embeddability.
That project is underrated.
I have built my agent harness around smolvm and it has been working wonderfully.
I have seen a single bug up to now (iirc you cannot take a snapshot from a machine you rehydrate from a pack) but found a workaround.
I think the only thing I would love a solution for is the ability to bind credentials even when you use tailscale. You cannot right now because the traffic goes through the tailscale interface.
Anyway. smolvm is simply amazing.
Like Docker Sandbox this meets what I think should be our standards in the AI age: kernel isolation and secret shielding in addition to network and disk isolation.
The things that are design choices for these tools are * agent integration * docker image composition * are vms more persistent or throw away * managing configuration
I have been working on a similar project because I haven't seen one before that to me didn't have design deficiencies. This project looks good to me but I need to try it out now and experience the design choices.
- a bunch of gitops repos: terraform mono repo, argocd-esk conf repo, grafana provisioning repo, etc.
- frontend + backend repo (tbh may merge these to a mono-repo, but here we are)
I think the auto detection is interesting, but tbh with my solution I've been happy with just install all the tools regardless of what a specific project needs - with a COW base layer this is cheap.
https://github.com/docker/sbx-releases
Mines based on libvirt and uses cloud-init for creating the images, with shared golden images across projects.
Skimming your docs it wasn't totally clear to me if your images are built per project, or COW from shared toolchains.
Looks like you have a few features in there that I might take some inspiration from though, eg the proxying stuff.
Funny how many of us are trying to solve the same problem at the moment.
My working threat model for my projects is less about "malicious agent trys to escape VM" and more "well meaning agent got confused and screwed up" - the added protection of a VM is desirable, and it's useful for stuff like nested virtualization (agent working on improving the VM orchestration), makes containers inside the sandbox straightforward
I hadn't seen the stuff about filtering spoofed packets so I'll probably add that.
Many of my projects use pnpm, which ideally uses hard links from a shared store of dependencies.
One of my goals is to reduce the disk overhead of VM based isolation, so I'd like a shared pnpm store across projects (and the same for maven, etc as well)
With https://virtio-fs.gitlab.io/ mounts, each mount inside the VM is a separate FS, and I can't hard link across them. I've tried playing around with ideas like bind mounting the directory structure from the host in a folder to have a single virtio-fs mount inside the VM - but the bind mounts still can't be hard linked across it seems.
I'm basically wondering if I'm missing something or just trying to do something that isn't possible.
All the host folders are on the same FS - but I don't want to organise the host system around the VM mounting needs, and I also don't want to mount extraneous directories into the VM (eg: simple solution could be just mount ~/projects but kinda self defeating)
(Fedora workstation host)
Happy to hear what other people are thinking. It seems like putting an intercepting HTTP proxy is getting popular in other projects.
I think at a high level I'm trying to treat them as junior/intern employees where I give them enough permission to do their role, but not enough rope to hang themselves.
See this user for example: https://github.com/filippo-agent
3. Account Requirements
We have a few simple rules for Accounts on GitHub's Service.
You must be a human to create an Account. Accounts registered by "bots" or other automated methods are not permitted. We do permit machine accounts:
A machine account is an Account set up by an individual human who accepts the Terms on behalf of the Account, provides a valid email address, and is responsible for its actions. A machine account is used exclusively for performing automated tasks. Multiple users may direct the actions of a machine account, but the owner of the Account is ultimately responsible for the machine's actions. You may maintain no more than one free machine account in addition to your free Personal Account.
One person or legal entity may maintain no more than one free Account (if you choose to control a machine account as well, that's fine, but it can only be used for running a machine).
I don't give it access to GH - it can read most things via API or web scraping fast enough and I manually run the commands I need outside the sandbox.
No need for SSH - it's running locally.
---
Given how many holes you're poking into the container I think you'd be better served with a real VM.
I guess they can add all the sibling comments to the list, as well as mine: https://github.com/AdrianVollmer/Agent-Sandbox
(Python, Podman, Squid for network filtering)
https://linuxcontainers.org/incus/
my main bottleneck was exactly in usability: how to make it least effort to spin up per use case and keep the set of operating scripts small - i didn't want to build an app for this, just shortcuts to use existing app.
This sounds good. Cloud-init is still worth the time, I think (as I just said in another comment)
I need isolation, but I also need to give my agents secure access to internal package registries, logs, internal infra, designs, emulators, browsers, docs etc. At a large company, work can span dozens of repositories. And on top of that, I need dozens of these.
I built this at work. Engineers use krunkit VMs with proper memory ballooning, GPU acceleration. Retina RDP, credential injecting proxy for etc.
This seems like a fun toy for single project development, but nothing close to what you need as an engineer in a corporate environment.
Because it's not just VMs, but a lot more around credentials injection, egress, volumes, agents selection policies. I am quite sure also the memory ballooning and GPU acceleration can be fine tuned.
In advance, thank you very much for your feedback!
A lot of work me and my fellow engineers do spans weeks if not months, tens of code bases, tickets etc. Having longer lived sandboxes that cover N repos in which you can just let experiments exist, together with plans etc is a superpower.
Eventually I can prepare a sandbox per repository or per "whole project" as we call it.
It's fascinating as a problem - thanks for sharing!!!
The problem i having is more workflow related. AI is doing the dev so fast I constantly need to validate different versions and change sets of an app. Git worktrees work for isolating the agents changes but doesn’t solve running multiple instances of a dev server and remembering which one applies to which agent session.
The solution I came up with was throw the code and all tools in a container git branch. Then run a reverse proxy to get <mybranch>.localhost style urls working.
Would be interested to give this a shot if it can also solve for some of those workflow issues.
https://github.com/canonical/workshop
I tried the remote SSH thing and it's painful to setup. Did anyone find a better solution than scripting a "create Docker container, bind the gpu, bindmount an external directory for data, install code server, etc."?
This is very true. Having been a user of heavily-customised vagrant boxes for a long time I've been spending a fair bit of time lately on this.
But my own stupid old-fashioned view is that it's probably worth learning cloud-init if you never have. Deploy your VMs on your own machine with multipass, same deployment works with little change in the cloud.
That's been the approach I have settled on, but only after a disappointing experience with Apple's Container Machine, which is both quite bothersomely insecure by default (your whole home directory? mounted rw? in a VM with an agent? really?) and from there sort of difficult to configure more effectively, and weirdly idiosyncratic where DNS/networking is concerned.
While running the Agent inside a VM/MicroVM means you're mainly exposed through potential hypervisor exploits, and potentially to vulnerabilities in your custom Host-Guest interface. Which makes it safer, once you've set it up, it covers a broader set of use case out of the box, unlike srt where you might have to keep changing your config file.
That's my experience after trying SRT and a few MicroVM setups though. I still use both, just for different stuff, but random code always goes straight into the MicroVM.