Seems like something that could also have enterprise applications for Shadow AI within an organization. Wouldn't be surprised if some of the observability/governance companies pick this up to use in their stack.
> One read-only command that inventories every AI agent, harness, MCP server, plugin, and AI extension on a machine
If this tool is returning even a single hit from this, you're probably using these agents wrong. You really want to run these in a way so they cannot touch your system drive/general filesystem that you use to do real work on. Even SOTA models at the end of their context limit behave REALLY illogical and does mistakes frequently. Don't run them straight on your machine unless you have backups and confirmed your backups work.
This is the correct response, but it also feels as though ≈nobody is sandboxing their agents/harnesses in practice. Or at least, a good majority isn’t.
I think there is a cycle, like with most things of this nature.
You start really locked down, and you read and approve every request for access. You do this for a while, but never see a result you deny, so you stop reading as closely. You keep hitting that approve button. You start paying even less attention to it. You start feeling silly, like you are simply slowing the process down. You get frustrated, because you keep coming back to your session and realizing your agent has been stuck waiting for approval for a long time, and the task that would have been done by now hasn’t even started.
Now you are running even more simultaneous sessions, which means more and more of your agents are stuck waiting for your approval. You feel even sillier, because you are taking even less time now to review and approve requests, but your review step is causing more and more slowdowns because you have more sessions going so you take longer between approvals. You feel like you are spending most of your time cycling through sessions hitting approve. You still have not come across a request that was dangerous or would have caused an issue, so you feel more and more like you are wasting your time.
So you slowly start to give your agents more access with fewer review steps.
Maybe this results in a catastrophic failure at some point, or maybe it doesn’t.
> You start really locked down, and you read and approve every request for access.
Or you start out with almost every permission granted always, but the entire thing runs in a sandbox with the minimum amount of credentials needed to do its work… Sounds preferable to me to catastrophic failures :-)
AFAIK, I know plenty of frontend engineers that without blinking run "npm install" on random 3rd party projects they find on GitHub, and they do their banking and everything else on the same machine. But then again, I also know people (one person to be fair) who have unprotected sex with prostitutes, so maybe something makes me slightly biased here.
I mean yeah, there are a ton of people who are stupid enough to disregard all kind of safety precautions. That doesn't mean the safety precautions are invalid though.
I run it on a physically dedicated machine, it has a root there (through sudo nopassword) and I keep some of my stuff there too. I don't do it for security reasons, I just want it to run when my laptop is closed and I'm outside or what not.
Haven't seen any close calls so far. The thing just behaves. Nothing of the horror stories of eremerefing the whole home directory or a database. Am I just lucky?
Aside from random errors made by the model, another reason to sandbox is that prompt injection could make your agent behave in an actively harmful manner.
Built my own harness with much better sandboxing, because I am using abliterated models for the implementations and they sure try to escape out of their sandboxes. A lot.
The sandbox from other harnesses like e.g. OpenCode essentially is useless. They think parsing arguments alone to detect paths helps, and LLMs know that because they will just pipe data differently then to escape the path traversal checks. Happened a lot to me (and was the primary reason for my to write my own harness from scratch).
Thinking in filesystem access alone is also a futile attempt at sandboxing. A much better way to think about sandboxes is policies and capabilities, with temporary access for agent roles rather than sessions (or, well, forever).
I’ve been looking along a similar line, but I came at it from the infra side rather than the software.
Mine pretends to be a local ACP agent, but it’s actually managing a Docker container and proxying the ACP connection into the container over websocket. You can specify a bunch of utility stuff in the YAML definition like directories to bind-mount, directories to copy from the host, scripts to run when the container starts, etc. You can also toggle whether ACP-native tools like read/write file and shells execute on your host or in the container (in container by default).
Works well, I forget whether I’m on that or pi-ACP directly until it spits out a time in the wrong timezone or I forget that it can’t check my DNS settings or something.
Really cuts down on the damage it can do. Mine is basically down to “it can delete my ~/.pi and the repo it’s working on” and that’s about it unless it can escape the container.
I need to wire a pixel-display to show emoji faces of the agents and how they roll their eyes and all of that. I have the display, I have the prompt to pick a mood from emotional vocabulary, a custom harness and all of that. I just can't figure out how to make the eyes move on a statically pre-rendered emoji.
34 comments
[ 4.0 ms ] story [ 57.0 ms ] threadIf this tool is returning even a single hit from this, you're probably using these agents wrong. You really want to run these in a way so they cannot touch your system drive/general filesystem that you use to do real work on. Even SOTA models at the end of their context limit behave REALLY illogical and does mistakes frequently. Don't run them straight on your machine unless you have backups and confirmed your backups work.
You start really locked down, and you read and approve every request for access. You do this for a while, but never see a result you deny, so you stop reading as closely. You keep hitting that approve button. You start paying even less attention to it. You start feeling silly, like you are simply slowing the process down. You get frustrated, because you keep coming back to your session and realizing your agent has been stuck waiting for approval for a long time, and the task that would have been done by now hasn’t even started.
Now you are running even more simultaneous sessions, which means more and more of your agents are stuck waiting for your approval. You feel even sillier, because you are taking even less time now to review and approve requests, but your review step is causing more and more slowdowns because you have more sessions going so you take longer between approvals. You feel like you are spending most of your time cycling through sessions hitting approve. You still have not come across a request that was dangerous or would have caused an issue, so you feel more and more like you are wasting your time.
So you slowly start to give your agents more access with fewer review steps.
Maybe this results in a catastrophic failure at some point, or maybe it doesn’t.
Or you start out with almost every permission granted always, but the entire thing runs in a sandbox with the minimum amount of credentials needed to do its work… Sounds preferable to me to catastrophic failures :-)
Haven't seen any close calls so far. The thing just behaves. Nothing of the horror stories of eremerefing the whole home directory or a database. Am I just lucky?
The trick is not go to that limit, but stay under 50% or even better 25% of context length. But backups are a smart thing anyway.
Built my own harness with much better sandboxing, because I am using abliterated models for the implementations and they sure try to escape out of their sandboxes. A lot.
The sandbox from other harnesses like e.g. OpenCode essentially is useless. They think parsing arguments alone to detect paths helps, and LLMs know that because they will just pipe data differently then to escape the path traversal checks. Happened a lot to me (and was the primary reason for my to write my own harness from scratch).
Thinking in filesystem access alone is also a futile attempt at sandboxing. A much better way to think about sandboxes is policies and capabilities, with temporary access for agent roles rather than sessions (or, well, forever).
[1] UI still needs lots of work though: https://github.com/cookiengineer/exocomp
Mine pretends to be a local ACP agent, but it’s actually managing a Docker container and proxying the ACP connection into the container over websocket. You can specify a bunch of utility stuff in the YAML definition like directories to bind-mount, directories to copy from the host, scripts to run when the container starts, etc. You can also toggle whether ACP-native tools like read/write file and shells execute on your host or in the container (in container by default).
Works well, I forget whether I’m on that or pi-ACP directly until it spits out a time in the wrong timezone or I forget that it can’t check my DNS settings or something.
Really cuts down on the damage it can do. Mine is basically down to “it can delete my ~/.pi and the repo it’s working on” and that’s about it unless it can escape the container.
https://github.com/SethCurry/abyss
Prepping for AI apocalypse I see