Just gonna toss this out there, using an agent for code review is a little weird. You can calculate a covering set for the PR deterministically and feed that into a long context model along with the diff and any relevant metadata and get a good review in one shot without the hassle.
OT: I wonder if WASM is ready to fulfill the sandboxing needs expressed in this article, i.e. can we put the AI agent into a web assembly sandbox and have it function as required?
If the agent only needs the filesystem then probably. If it needs to execute code then things get flaky. The WASM/WASI/WASIX ecosystem still has gaps (notably no nodejs).
This is a good explanation of how standard filesystem sandboxing works, but it's hopefully not trying to be convincing to security engineers.
> At Greptile, we run our agent process in a locked-down rootless podman container so that we have kernel guarantees that it sees only things it’s supposed to.
This sounds like a runc container because they've not said otherwise. runc has a long history with filesystem exploits based on leaked file descriptors and `openat` without NO_FOLLOW.
The agent ecosystem seems to have already settled on VMs or gVisor[2] being table-stakes. We use the latter.
chroot'ing isn't sandboxing or "containers". And I don't think it's a very good explanation, actually - not that its necessarily easy to explain.
It looks like the author just discovered the kernel and syscalls and is sharing it - but, it's not exactly new or rocket science.
The author probably should use the existing sandbox libraries to sandbox their code - and that has nothing to with AI Agents actually, any process will benefit from sandboxing, that it runs on LLM replies or not.
The seems to be looking to let the agent access the source code for review. But in that case, the agent should only see the codebase and nothing else. For a code review agent, all it really needs are:
- Access to files in the repositorie(s)
- Access to the patch/diff being reviewed
- Ability to perform text/semantic search across the codebase
That doesn’t require running the agent inside a container on a system with sensitive data. Exposing an API to the agent that specifically give it access to the above data, avoiding the risk altogether.
If it's really important that the agent is able to use a shell, why not use something like codespaces and run it in there?
Containers might be fine if you’re only sandboxing filesystem access, but once an agent is executing code, kernel-level escapes are a concern. You need at least a VM boundary (or something equivalent) in that case.
10 comments
[ 2.7 ms ] story [ 22.9 ms ] thread> At Greptile, we run our agent process in a locked-down rootless podman container so that we have kernel guarantees that it sees only things it’s supposed to.
This sounds like a runc container because they've not said otherwise. runc has a long history with filesystem exploits based on leaked file descriptors and `openat` without NO_FOLLOW.
The agent ecosystem seems to have already settled on VMs or gVisor[2] being table-stakes. We use the latter.
1. https://github.com/opencontainers/runc/security/advisories/G...
2. https://gvisor.dev/docs/architecture_guide/security/
It looks like the author just discovered the kernel and syscalls and is sharing it - but, it's not exactly new or rocket science.
The author probably should use the existing sandbox libraries to sandbox their code - and that has nothing to with AI Agents actually, any process will benefit from sandboxing, that it runs on LLM replies or not.
"Early civilizations had no concept of zero..."
- Access to files in the repositorie(s)
- Access to the patch/diff being reviewed
- Ability to perform text/semantic search across the codebase
That doesn’t require running the agent inside a container on a system with sensitive data. Exposing an API to the agent that specifically give it access to the above data, avoiding the risk altogether.
If it's really important that the agent is able to use a shell, why not use something like codespaces and run it in there?