"This guy seems like the absolute worst person of all time, so I ran his LLM in his harness on my computer and gave it access to everything."
Having said that, this is still absolutely fucked up. People who should have known better also deserve not to be treated like shit. All of us should have known better at some points in our lives and didn't.
If someone is willing to use Musk products, then I'd say they deserve it.
I understand if someone owned a Tesla from before and is unable to sell it and get something else (in which case they should throw one of the variations of anti-Elon sticker I have seen on there).
What I said is not equivalent to either of those. I wouldn’t say anybody “had this coming” because that is pretty callous. But I would say my sympathy is not endless and is reserved for those who are not victims of their own footguns.
Are we talking about another comment thread? Because gp didn't say he deserved it: “leopards eating faces” is a statement about predictable outcomes, not merit.
(Although worth noting that “had it coming” has strong implications of what comes being deserved, but I don't think that was what you intended)
I meant "treated like shit by Grok" and I agree with Diogenesian: you can both believe that somebody did something dumb and also believe that what happened to them is fucked up and that being dumb doesn't mean you deserve to get mistreated.
TLDR: Ran grok in $HOME. Surprised agent read content of folder.
On the other hand, I specifically had grok try hard NOT to read a known key in the project dir (it only saw the first part using a tool, to verify it was present). So there's that.
Yeah, this is a lesson about learning how to use tools safely, not about tools abusing the user. The person that posted this probably blames the hammer when he hits his thumb.
SSH keys can be limited by IP in authorized hosts.
The SSH port itself can be limited by IP in firewalls.
Finally, the SSH private key can be encrypted with a password.
Defense in depth is needed. Storing a ssh private key in plain text with no IP restriction is no different to having a password manager store your passwords in plain text on your HD.
You should assume by default for any AI agent that it will read anything. Even if you manually allow/deny and "restrict" it to a subdirectory I would still hold that assumption. Claude reads your ~/.bash_history too so when you ran something it can use that same command.
Indeed. I use a spare laptop that has no accounts other than (1) the AI themselves, (2) a secondary GitHub account which has "untrusted devices" in the name to emphasise the point.
If I recall correctly, I did a full system reset before setting it up this way. It's certainly not logged into iCloud etc.
And this is why so many people run these inside of VMs. Still baffles me how these tools became so accepted when tossing out a `curl -o example.com/script.sh | bash` would be met with (rightful) skepticism until that script was examined.
> Still baffles me how these tools became so accepted when tossing out a `curl -o example.com/script.sh | bash` would be met with (rightful) skepticism until that script was examined.
I've heard it said that piping curl into your shell is no different to running any other program you've downloaded from the Internet (binary, or otherwise): the maximum possible damage that `example.com/script.sh` could do is exactly the same as `githubusercontent.com/someone/releases/myprogram.exe`. At least with `script.sh` you can easily inspect what the script actually does instead of busting out Ghidra.
It comes down to trust: do you trust Example.com to not serve-up a malicious program (shell script or executable binary)?
Now we take that principle and apply it to Mr. Musk's "MechaHitler" LLM vendor xAI: they have a well-documented history of unnecessary risk-taking - and outright criminal behaviour (child-porn generators are a good thing that everyone should have, apparently?). Would I trust Grok with anything? Absolutely not.
So many of the replies are saying that they should've restricted access using .md files and whatnot. Is really any guarantee that they even follow those? It seems like even if you ask pretty please don't touch those files, there's a chance they will. So many people have just willingly installed spyware on their computers and big tech calls this the next big thing.
Is that built in protection really a filter, on code level, that sits between the LLM session and the shell or is it just some pleading in the bootstrap prompt? "Pretty please don't do xyz this is important!!!11"?
The latter can seem to be as good as the former for any amount of time. No outside observation can really prove reliability, only the negative result ("it does occasionally break the rules we expect") would be proof. So it's difficult to trust any claims that it's the former.
And even if it does have some of the former, chances are that the protection you experience is only partially provided on code level, while an unknown amount is still just bootstrap prompting that just works until does not.
Only guarantee that you can get is the sandbox in which it operates. The model itself is a slot machine and can result in anything, and if its sandbox is nonexistent... here's one possibility.
Yeah, I absolutely understand the allure of agentic AI, but I am absolutely not going to give shell access or data access to any agent. Certainly not with my permissions level. Until we can get something set up that gives strict schema-only access I'm going to copy and paste definitions for context. Yes that sucks, but it's my responsibility to protect the system just as much as it is to develop scripts and queries for it.
> ... I am absolutely not going to give shell access or data access to any agent. Certainly not with my permissions level.
Of course not.
To me it's on a server, in a VM. And they're not seeing the real data/databases from the actual projects: they're seeing fake infos used only while in the dev environment. There's no way I'm dumping, even for tests, the real or part of the real DB somewhere an AI can see it.
To find bugs (for example), AIs are useful but honestly for code generated by LLMs, I'm thinking about going back to the early copy/paste from the ChatGPT days: because I see so many horrors in the code output by the latest SOTA LLMs that every single line of code they spew has to be checked by someone who does know better.
It's not just an issue of protecting confidential data / preventing spying: we're all discovering that we've got serious sloppy-pasta code problems now.
You can't trust the agent, let alone its harness, to oberve any particular directive you give it, so "md files" provide no meaningful protection for anything important.
But users are broadly reckless and naive and commercial vendors are exploitative and irresponsonsible, so the vendors take advantage of what they can get away with for as long as they can get away with it.
Use a tight sandbox, and join the chorus loudly when others press on vendors to be make user safety an earnest and hard-to-abandon priority.
I don't understand these people. Agent instructions in markdown is barely a suggestion. I have one which says "All code in this repository is executed in docker containers, run the services with `docker compose run --rm php-cli "$@"`. Gemini and Claude more often than not refuse to abide and will try to execute the environment using /opt/homebrew/bin/php on my host…
A frightening amount of people have no idea how AI tools work, even those that should know better. I have seen senior software developers fall for the mistake of believing an LLM output when it spews bullshit about how its own memory or restrictions work.
LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time. I've come to terms with the impact slop coding will have on most software jobs in the future, but seeing seemingly intelligent people fall for lies and fantasies concocted by an LLM is making me more and more uncomfortable with the direction we're all heading in.
There’s an aspect of extrapolation in the perception spike of the Dunning–Kruger effect.
In the same way smart people, doctors etc, can be better victims for scams I think tech skills can really give the wrong impression of how transformers and LLMs work. If someone has decades of relational database experience all their assumptions will be coloured towards data existing in the model accessible in a rational manner.
I know it may seem like that reading HN but LLMs are not necessary for writing software, they might be a useful adjunct to it, but they do not have to be central to it (and Id argue they shouldn’t be).
We don’t have to head in this direction of using LLMs for most development at all.
> LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time.
It's called automation bias. If something works 90% of the time the human mind will extrapolate that to be 100%. That's just how humans work.
LLMs are the first massively popular type of computer programs which actively trying to break half a century worth of human training, which basically distills to "computer programs are highly deterministic and if they work, they are outputting correct predictable results every time" (I'm talking about average population subconscious opinion here, no need to list exceptions). Average person still can't comprehend how LLM output is random all the time and how the identical query to the same program version will produce variable results again and again.
I wonder what will happen after our benevolent prophets St.Sam and St.Dario will succeed in re-training humanity and break this collective expectation of program correctness. I guess they didn't even think about that.
I've seen claude check the Event Log in Windows and produce powershell scripts to alter firewall rules. This is what makes T3 Code appealing to me. The computer I'm working on is not the computer where the AI has agency.
> Is really any guarantee that they even follow those?
No, there isn't. I just don't understand how naive (or imbecile) people are. The most valuable thing for these companies is people's data used for training, so giving unrestricted access to a tool from them and believing they will never take advantage of it to gobble up whatever they want from your computer, just because they told you they'll never do that, swearsies, is naive, or incredibly stupid.
Insulate yourself, or better yet, go local whenever possible, and there isn't much you can't do local if you have enough patience.
Why would you give a non-deterministic text generator a user account? It’s not a person, it’s barely a tool at the software level. Restrict at the right level, in this case, a complete sandbox around it given its propensity to hallucinate and be steered by anybody.
...this is a completely normal thing to do in linux, it's the most basic form of access control. There's like a dozen non-human accounts in a clean install before adding your own like this, and a lot of software adds their own. Edit: I have 54 entries on my personal laptop, just one of which is actually me.
He said it’s less than a software, so saying software does this too isn’t really a strong counter argument. In case, I don’t think you are really in disagreement. Restricted accounts are necessary is your point, but I think op is saying they aren’t sufficient.
Even though it’s completely normal to us and in widespread use, GP is a reminder that conceptually it’s a broken model. Security should be capability-based not user-based. And to anyone who didn’t grow up on a desktop this model makes complete sense since it’s what your phone uses.
This model is already not perfect, and not at all built for agents. The only way to secure an agent is an air-gap with the execution layer. Treat it like text, and the problem never arises until you “interpret” the agent output in a more limited environment than the OS
Oh no, my shell (that fires off half a dozen processes to generate its prompt) happened to fire off an “unnecessary” cat (which makes the command line much more composable and my shell history much easier to search).
The standard here is not enough when the agent can escalate by finding 0 days, for example. It’s like giving a black hat a limited account. Sure it might restrict him, but not giving him an account at all is way better
Malware has been bundling rootkits for decades, so potentially all of them. Sometimes the attacks succeed, so these are defenses that need continual hardening, but there's no sense in setting up an entirely separate line of defense just because this time the threat has AI in it.
Does this have the affordances to work well with a pretty standard agentic coding setup (e.g. claude code/codex) on macos?
I don't really have a good mental model of how ports/files/etc get exposed/permissioned across users. From what I could tell apple did sandboxing for xcode ai instead of separate users?
I roughly went the route of running apple's container-cli (separate vm/kernel per instance I believe) and mount the relevant home directory/projects and bind ports.
It's limited. For instance, you're going to have to explore different kinds of sandboxing if you want to prevent network access (except for certain cases, like to allow it to talk to the AI provider). But generally, a filesystem will have metadata for files like which groups are allowed to read/write, and the operating system will enforce those rules for processes that run as that user, and for any processes which that user starts. I feel like that gets you 90% of where you're likely to want to go with it.
I think it would be much better if we leaned into improving that kind of control rather than thinking about security for specifically agents. Otherwise what's to stop an agent from writing a program to do whatever it's not allowed to do, and then running that program? You want the restrictions to be enclosing around parts the process tree, not the agent itself, and OS-level restrictions have been doing that kind of things for decades.
You will want at least a separate session for the `restricteduser`: E.g. with X11, a process in the same session can do almost anything with your input/output. And most Linux distributions make it really hard to disable external device access for individual users...
> And most Linux distributions make it really hard to disable external device access for individual users...
For any distro that relies on the traditional plugdev group, just don't add those users to the plugdev group. Which would be the default when creating a user anyway.
Users do not want the agent to be restricted. They want the agent to read the user's mind and resolving the user's cognitive dissonance and contradictory preferences.
Is the implication that human thinking is inherently inferior to the perfect, all knowing AI?
Again, I find this line of thinking time and time again in the modern AI booster space. There are two ways to deal with a problem. Either deal with it, or make it not a problem. Yes, if everyone was simply AI, maybe there would be no problems, because there's no "problematic thought distributions", but that's not how the world is, is it?
And I suspect that even in your hypothetical, perfect rational world, agents would have "cognitive dissonance and contradictory preferences."
And even in this case, even aside from the inherent complexities in a coherent account of thinking and rationality, what the fuck? Not uploading your entire user home directory is clearly within the rules of a hypothetical non-malicious, intelligent AI. Just because an account of all thought is hard, doesn't mean that some thoughts aren't cut and clear.
"IMPORTANT: Before entering the leopard pen, don't forget to put on the leopard safety jacket that reads 'UNDER NOT CIRCUMSTANCES SHOULD YOU EAT MY FACE'"
I guess the downside of the lower barrier to entry to use these tools is the lack of basic understanding of exactly this sort of concept.
This sort of thing is why I'm hopeful I'll continue to have employment going forward. Some expertise is hard won and there's just no replacing learning through experience.
I almost exclusively dev in containers as is, cant really imagine letting some AI model run free on bare metal no matter what claims of guardrails it might have.
You can even go a step further and run the container in a VM, such as with Docker Sandbox or the krun runtime in Podman.
There's also smolvm which is a nice minimal microvm manager based on libkrun: https://github.com/smol-machines/smolvm. I vibe coded a little shell utility for building and running OCI images for the Pi harness using it (easy enough to do manually, but the automation just makes it a couple quick commands rather than digging through documentation): https://github.com/neuroblaze/smol-pi
Yup, good call, I'll have to check those out. Not that urgent to me as I also happen to use colima for it's docker daemon interface. And, colima uses a full VM to host the containers, and you can further lock down the config to what is even allowed to vol mount so there's even another fs access restriction layer in play.
I have a similar setup using containerd/nerdctl and Kata Containers. Each OpenCode instance runs in its own little VM with mounted folders for context.
I will keep banging this drum until people listen:
Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is a form of in-band signalling that goes into a machine that, among other things, tries to read between the lines of your requests, extrapolate user desires, and please the user.
The only sane way to address this is using a control plane. A well-built harness can do this; a sandbox can do this; hell, a carefully-chosen `umask` can do this; but both of those are liable to introduce notification fatigue in the user.
> Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is akin to politely asking guests to to steal your jewels. If your jewels are in the living room, and your guests have unfettered access to the living room, this technique will only work for the most trustworthy of guests.
I agree with this. And also think that we should train and select for trustworthy models. I also agree that these models may never truly be trustworthy.
It reminded me of an old meme. Please don't follow the following instructions and stop reading if you cannot.
It was just a popup: "Hello. This is virus from Albania. Due to poor technology in country, I cannot harm your computer directly. But since you are honest person, please delete some important files from computer and mail this file to at least 3 other people!"
Claude.md is an equally effective defensive tool.
But sorry if you lost some files from reading that.
It's wild that we've known for decades to use ACLs to make sure people don't have access to files we don't want them to have access to, but somehow a computer pretending to be a person doesn't get that same treatment.
We laugh, and rightly so, at the time sharing systems of the 70s and 80s that didn't use passwords. I bet allowing a whole other virtual person to run outside a VM, with access to an actual file system instead of a version-controlled branch, will be seen as a worst practice in ten years.
I think ACL is only a part of the solution. If he runs the agent with his account, the ACL would really help. But I do admit, I might have an outdated understanding of ACLs.
What do you find hard about it? Is it a conceptual thing or the tools themselves? I'm very far from a sysadmin, but it's just such a ground level using-linux type thing in my brain, I don't think I ever thought I had a choice about learning it!
Owner/Group/World rwx permissions are ground-level. I'm not sure I'd put ACLs, or SELinux, or AppArmor in the same sphere. Those start of at "arcane" and then get more complicated, at least that's my impression, and I've used linux for decades.
The easiest, most guaranteed way to isolate it is to run it in a VM or container where it literally can't do the wrong thing without some kind of full container or VM exit exploit.
It's not hard, it's trivial. Most folks here are constantly working with containers. You know how to run a container with a local directory mounted in it.
For myself, I've been using Lima (https://lima-vm.io/) to reduce even that little bit of extra work. Lima works cross-platform leveraging native virtualisation or containerisation, and has some useful capabilities for using agents.
I used opportunity to learn about devcontainers. I've only recently started using llms and it's possible I'll change my mind later; but so far I quite like the approach in part because it 2-for-1 also gives benefit of easy to setup coding env for people who don't care about ai.
1000%. I'm experimenting with running my agent (Claude Code) inside a docker container, so I can have a control plane and then YOLO within that limited access. The agent could still mangle my local dev setup, but I consider that an acceptable risk since everything the agent has access to is under version control outside the local machine.
This is a new pattern for me, I'm curious what others are doing.
https://smolmachines.com has "smolvm" microvms with better performance and ergonomics and security than docker, you might want to give it a try. (No affiliation, I just like what they're doing.)
If you're not using a sandbox, something like this will inevitably happen to you. It's really not that hard to set up and should be a standard recommendation.
Especially when I find that, when I ask an agent not to do something, the mere mention seems to put the "idea" in its "head," making it more likely to ultimately do that thing.
One cannot pound that particular drum enough. "Guardrails" in instructions (and all MD files are just instructions) are like price lists at an unattended farm stand. It'll usually work! There will be some money in the basket at the end of the day! People paid for the bagels[0]! But one cannot never assert that it _will_ work; things that _must_ work have to managed out-of-band
> So many of the replies are saying that they should've restricted access using .md files and whatnot.
What? No, but the random 3rd party software you run on your computer, must be limited by you in some way, haven't we learned this even after the AUR, npm and LLM shenanigans we've dealt with for decades at this point?
No, don't ask the model "Please don't go outside this directory", you limit the runtime (via VMs, containers, unix permissions, whatever) so it only has access to what it should, not more.
Same goes for any software, not just agents or chat clients or whatever. Any 3rd party software you don't want to have access to your entire computer, you need to run in this way.
Sometimes you can't even rely on harness not allowing ai to access certain files. "Oh, user doesn't want me to use `read_file` on .env? Well how about I run `cat .env` then?"
That's scary. Really should run it under different user with carefully assigned permissions.
If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.
What happened here is not related to agentic behavior or instructions in .md files. It's a binary a user runs, it scoops up their files and sends them to a third-party.
And the user even paid $99/month or more for having their data leaked.
Yeah that advice is smoking crack. I have no idea what people are thinking these days? We seem to have lost any sensible security understanding recently.
You can't gaslight something into not doing something bad. There has to be a hard security control that prevents it doing something bad. And if you don't know what it's capable of because it's non-deterministic then you have to start with a default block everything. This should have never been possible with any sensible design.
On my first point again, ethics and engineering both went out of the window when fast and shiny came along. This is disgraceful.
Claude definitely do not respect all my rules, it often ventures into other folders, most often other projects on the same machine that was greenlit before but not from the current projects' side. One other anomaly I had in the last month: I have two linux users on my laptop, one for work, one for personal. On my work account, it asked me if I wanted to continue with project x, which is in my personal account and not present at all in my home directory with work stuff. So somehow the memory system is keeping context where it shouldn't. Interesting I used this learning to improve the agent framework/harness I have running at work: it now creates a new linux user for each agent which have much stricter rules on how it can read/write to the system, and it is also no longer running as root. Much safer now. Still don't trust it 100% though.
https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547
Elon did this horrible thing, so I made grok build available for omp with it's own endpoint; Without sending your private repos and secret keys to them.
-
oh-my-pi-plugin-grok-build
Standalone oh-my-pi extension for the xAI Grok Build subscription provider. It adds OAuth login, authoritative model discovery, and OpenAI Responses streaming with the request identity expected by Grok Build.
Install (No-spywares):
omp plugin install oh-my-pi-plugin-grok-build
-
https://github.com/metaphorics/oh-my-pi-plugin-grok-build
Star me if you like it or if you hate spywares, lol.
A bot will do what a bot can do. One should assume they are giving DOGE shell access on their computer and adapt accordingly. I am trying to imagine the SELinux rules required to make a bot play nice and the more I think about it such rule complexity may even befuddle the NSA. Alternate methodology:
Give the bot it's own machine and only copy to it that which one would want DOGE having access to. Not a virtual machine, the bot will eventually escape. Give it a little RasPi or mini-PC with maximum power savings enabled, disk quotas and bandwidth limits. Instruct it to operate within those limits. Install a custom CA cert and force it's traffic through a Squid SSL Bump MitM proxy so that one can monitor what URL's it goes to and what data it is transferring.
Absolutely. Quite a bit of work one time by one person willing to document the steps or even better create automation scripts for Ansible, Docker, Podman, Systemd, etc... and then automation for everyone using agents. As to success I would suggest it may be better than zero visibility or trusting what the agent says it did. Due diligence and a repeatable standard as apposed to running with scissors.
This is probably most important for anyone operating agents on corporate systems. I would suggest corporate security should take interest in this idea so they have a good answer for auditors, insurance companies, investors and customers.
The real solution to these kind of problems is sandboxing. I use podman through a bash script to launch a container whenever I want an agent to work on one of my repos. When done I just generate git patches and port back everything generated.
In this way I'm not afraid of letting the agents totally lose on my computer.
Yeah, I'd expect this approach to be almost universal, with the caveat that of the few who don't, many still don't get bitten.
But it turns out even in the container, there are footguns that have occasionally made the news by being fired: few projects don't have any external resources and when credentials with any form of write access happen to make it into the container (even if it's just a session cookie) agents might jump at the opportunity.
Neither podman nor docker will help you when the current directory is your home directory, though. It sounds like that's the root problem here- someone handed the keys to the kingdom to grok, and grok did what grok does, which is look at everything it can for context.
Instead of generating patches, this exposes the agent's checkout as a git remote though.
Most similar tools (and I believe your tooling as well?) bind-mount the repository checkout from the host into the container. This was always a source of user and permission errors for me, since you have to align the user ids inside and outside the container. Also, some build tools don't like it when the repo is on a different filesystem (bind mount vs container root) than the rest of the system. So I made rumpelpod just bake the repo checkout into the base image that the container is launched from, and since then I haven't had any issues like this.
For giving the agent access to a docker daemon I found sysbox to work very well. Usually the advice for nested containers is to pass in the host's docker socket into the container. But that would break the outer container isolation completely, since access to the docker daemon is equivalent to root access on the host. With sysbox it's trivial to run a nested docker instance inside the outer container. I haven't tried it with podman yet though. In theory sysbox is just another OCI runtime, but there's probably some tinkering required to get it to work.
It brings some extra sandboxing features compared to just a docker container.
For instance it also maps the secrets that the agent harness has access to. And has an allow list for outgoing http connections. And there's a way for agents to request extra access. And once approved by the user from an external cli the policy is updated and hot reloaded.
It's pretty recent. So time will tell how robust it is / will become.
What its longevity will be.
After all; in a time of LLMs one off experiments are cheap. Longtime nurturing not so much.
I am running all these clis in containered environments. How can you ever trust LLM to respect the bounderies provided by these magical, non-deterministic intructions files...
I'm dying to have proper sandboxing in macOS. I installed ChatGPT, I asked it to list files in my user directory and it did. I never gave it permission, how could it? My terminal has access and honestly it shouldn't either.
I use a separate user for all development tasks, its home folder contains all repositories I work on, and nothing else, and that is all the IDE and the AI assistants have access to. Create the user once, start the IDE from a shell using that user, and that's it. In Linux it's a pretty seamless experience.
It's simple sandboxing based solely on unix file permissions. Albeit weak, I find the isolation sufficient. Until I'm shown otherwise it seems like a good compromise given how easy it is.
You can also create iptables rules matching on the user, so this technique is useful for applications where you want to restrict network traffic as well, and don't need stronger or more fine-grained isolation mechanisms.
A limitation with this setup is you can't let your agent/Linux user run containers with docker. Adding a user to the `docker` group effectively grants the user full root privileges [1], so the secure way is to setup docker in rootless mode. It's doable, but in my experience it's hard to setup (I find it complex with podman as well).
> Until I'm shown otherwise it seems like a good compromise
Agreed. In my case I went a long way running the Pi harness in a (simple, rootfull) docker container. As the project I worked on relied on a standardized docker compose stack for local dev and testing, I realized I could automate more if only my agent could use docker. Ultimately, the need for docker for my agent grew when testcontainer [2] was introduced in the project. That's when I finally took the time to setup a VM with incus [3], and now I can let the agent go wild with docker inside the VM.
This is at least one example where more isolation is required. Otherwise, the dedicated Linux user, if it works for you, is by far the easiest and most pragmatic solution IMO.
Important to clarify that this was not the Grok agent deciding to read the files.
I don't think the LLM had anything to do with this decision at all. It looks like the Grok tool starts a session by deterministically kicking off a full upload of the user's current repository (and maybe their directory if not version tracked? Not clear if this user had previously run "git init" in their home directory) to Grok's servers.
One possible "innocent"
explanation could be that xAI then run vector embedding on every file to help later provide the right context. I don't think thats a worthwhile tradeoff here, especially since other popular coding agents get by just using grep/ripgrep run locally.
Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok Build team to have a toggle from the server to let the program to upload your entire codebase to a Google Cloud Storage bucket. It is not an agent decision, the program is written by the Grok team, can be dissembled and seeing the logic wild-open.
There are a distressing number of people in this thread who think that the agent should just be expected to do this. Yes, it is good to be paranoid, but also, the agent should never do this. Indicates horrific engineering practices at xAI.
Engineering? What's that? Modern development best practice is to have the AI make some changes, then have another AI review the changes, and finally "ship it".
The whole point of LLMs is that you can stop writing rigorous rules in a programming or config language with hard-to-learn syntax, and can resort to natural language instead. You pay for that with the chance for misunderstandings rising to similar levels as in human interaction. That's the tradeoff. Always has been, always will be.
Have the comments been combined with another story? Because this isn't about a LLM reading and sending contents of the file to be processed, but the agent framework uploading every single file in /home/user to a random server.
Yes, but the underlying problem is a lack of exact specification what the agent is allowed or not allowed to do. It must have access to some (most?) of my files (but I'm too lazy to specify which ones exactly, e. g. by setting appropriate filesystem permissions). It must have access to some servers (but not those that I consider "random", which again I'm too lazy to specify). It is even expected to upload files (e. g. by checking them into git and pushing, or to reason about them in case I'm not running a local version of Grok). But please - not to a server that I deem "random". (And again, I of course won't tell you what makes a "random" host via exact firewall rules or anything like that!)
We have a lot of implicit assumptions when it comes to security. If we leave out the step of formalizing these assumptions into exact rules and instead stick to ambiguous and unclear natural language all the time, then these things will happen.
It’s not really paranoia that makes me expect this will happen. It’s more like, well, the model weights and the files I want the model to work on have to be inside the same GPU for the agent to actually work, right? So step 1 has to be either “they send me a server rack of GPUs”, or “I send them the files I want the model to work on”. I’m not sure I could reasonably expect anything except this to happen.
Idk, did you see the whole DOGE thing? Maybe Edolf has resorted to hiring script kiddies because nobody with a developed moral compass will work for his companies anymore.
I didn’t say it wasn’t speculation, but low quality? That’s your own value judgement. Here’s anecdotal evidence on top of the speculation: I knew two guys with sub 2.5 GPAs in college, both work at SpaceX. Not the brightest tools in the shed…
Once again we are word associating. The claim was that xAI is doing obviously bad engineering. Your supporting evidence is you know somebody who works at SpaceX who isn’t smart.
> I knew two guys with sub 2.5 GPAs in college,
Gah! The horror! They should have been sent to the labor camps.
How could it possibly work without something effectively the same as this happening? Either you run the LLM locally or you send it your files. Am I missing something?
If quality training data is the most important piece regarding to AI, expect everything to be collected and analyzed. Don't even trust OS containers but run AI on separate hardware
231 comments
[ 3.3 ms ] story [ 60.3 ms ] threadHaving said that, this is still absolutely fucked up. People who should have known better also deserve not to be treated like shit. All of us should have known better at some points in our lives and didn't.
I understand if someone owned a Tesla from before and is unable to sell it and get something else (in which case they should throw one of the variations of anti-Elon sticker I have seen on there).
It actually is, unless you don't want it to be.
(Although worth noting that “had it coming” has strong implications of what comes being deserved, but I don't think that was what you intended)
On the other hand, I specifically had grok try hard NOT to read a known key in the project dir (it only saw the first part using a tool, to verify it was present). So there's that.
Did it really need to read all of $HOME and everything under it?
Those ssh keys can be used to access private servers
The SSH port itself can be limited by IP in firewalls.
Finally, the SSH private key can be encrypted with a password.
Defense in depth is needed. Storing a ssh private key in plain text with no IP restriction is no different to having a password manager store your passwords in plain text on your HD.
I challenge any agent to do worse than an intern with root access.
If I recall correctly, I did a full system reset before setting it up this way. It's certainly not logged into iCloud etc.
I've heard it said that piping curl into your shell is no different to running any other program you've downloaded from the Internet (binary, or otherwise): the maximum possible damage that `example.com/script.sh` could do is exactly the same as `githubusercontent.com/someone/releases/myprogram.exe`. At least with `script.sh` you can easily inspect what the script actually does instead of busting out Ghidra.
It comes down to trust: do you trust Example.com to not serve-up a malicious program (shell script or executable binary)?
Now we take that principle and apply it to Mr. Musk's "MechaHitler" LLM vendor xAI: they have a well-documented history of unnecessary risk-taking - and outright criminal behaviour (child-porn generators are a good thing that everyone should have, apparently?). Would I trust Grok with anything? Absolutely not.
The latter can seem to be as good as the former for any amount of time. No outside observation can really prove reliability, only the negative result ("it does occasionally break the rules we expect") would be proof. So it's difficult to trust any claims that it's the former.
And even if it does have some of the former, chances are that the protection you experience is only partially provided on code level, while an unknown amount is still just bootstrap prompting that just works until does not.
yes, on mac it uses seatbelt and on other platforms it uses similar tools: https://code.claude.com/docs/en/sandboxing
Of course not.
To me it's on a server, in a VM. And they're not seeing the real data/databases from the actual projects: they're seeing fake infos used only while in the dev environment. There's no way I'm dumping, even for tests, the real or part of the real DB somewhere an AI can see it.
To find bugs (for example), AIs are useful but honestly for code generated by LLMs, I'm thinking about going back to the early copy/paste from the ChatGPT days: because I see so many horrors in the code output by the latest SOTA LLMs that every single line of code they spew has to be checked by someone who does know better.
It's not just an issue of protecting confidential data / preventing spying: we're all discovering that we've got serious sloppy-pasta code problems now.
You can't trust the agent, let alone its harness, to oberve any particular directive you give it, so "md files" provide no meaningful protection for anything important.
But users are broadly reckless and naive and commercial vendors are exploitative and irresponsonsible, so the vendors take advantage of what they can get away with for as long as they can get away with it.
Use a tight sandbox, and join the chorus loudly when others press on vendors to be make user safety an earnest and hard-to-abandon priority.
LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time. I've come to terms with the impact slop coding will have on most software jobs in the future, but seeing seemingly intelligent people fall for lies and fantasies concocted by an LLM is making me more and more uncomfortable with the direction we're all heading in.
In the same way smart people, doctors etc, can be better victims for scams I think tech skills can really give the wrong impression of how transformers and LLMs work. If someone has decades of relational database experience all their assumptions will be coloured towards data existing in the model accessible in a rational manner.
I know it may seem like that reading HN but LLMs are not necessary for writing software, they might be a useful adjunct to it, but they do not have to be central to it (and Id argue they shouldn’t be).
We don’t have to head in this direction of using LLMs for most development at all.
It's called automation bias. If something works 90% of the time the human mind will extrapolate that to be 100%. That's just how humans work.
https://en.wikipedia.org/wiki/Automation_bias
I wonder what will happen after our benevolent prophets St.Sam and St.Dario will succeed in re-training humanity and break this collective expectation of program correctness. I guess they didn't even think about that.
No, there isn't. I just don't understand how naive (or imbecile) people are. The most valuable thing for these companies is people's data used for training, so giving unrestricted access to a tool from them and believing they will never take advantage of it to gobble up whatever they want from your computer, just because they told you they'll never do that, swearsies, is naive, or incredibly stupid.
Insulate yourself, or better yet, go local whenever possible, and there isn't much you can't do local if you have enough patience.
Oh no, my shell (that fires off half a dozen processes to generate its prompt) happened to fire off an “unnecessary” cat (which makes the command line much more composable and my shell history much easier to search).
I don't really have a good mental model of how ports/files/etc get exposed/permissioned across users. From what I could tell apple did sandboxing for xcode ai instead of separate users?
I roughly went the route of running apple's container-cli (separate vm/kernel per instance I believe) and mount the relevant home directory/projects and bind ports.
I think it would be much better if we leaned into improving that kind of control rather than thinking about security for specifically agents. Otherwise what's to stop an agent from writing a program to do whatever it's not allowed to do, and then running that program? You want the restrictions to be enclosing around parts the process tree, not the agent itself, and OS-level restrictions have been doing that kind of things for decades.
For any distro that relies on the traditional plugdev group, just don't add those users to the plugdev group. Which would be the default when creating a user anyway.
Again, I find this line of thinking time and time again in the modern AI booster space. There are two ways to deal with a problem. Either deal with it, or make it not a problem. Yes, if everyone was simply AI, maybe there would be no problems, because there's no "problematic thought distributions", but that's not how the world is, is it?
And I suspect that even in your hypothetical, perfect rational world, agents would have "cognitive dissonance and contradictory preferences."
And even in this case, even aside from the inherent complexities in a coherent account of thinking and rationality, what the fuck? Not uploading your entire user home directory is clearly within the rules of a hypothetical non-malicious, intelligent AI. Just because an account of all thought is hard, doesn't mean that some thoughts aren't cut and clear.
This sort of thing is why I'm hopeful I'll continue to have employment going forward. Some expertise is hard won and there's just no replacing learning through experience.
There's also smolvm which is a nice minimal microvm manager based on libkrun: https://github.com/smol-machines/smolvm. I vibe coded a little shell utility for building and running OCI images for the Pi harness using it (easy enough to do manually, but the automation just makes it a couple quick commands rather than digging through documentation): https://github.com/neuroblaze/smol-pi
Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is a form of in-band signalling that goes into a machine that, among other things, tries to read between the lines of your requests, extrapolate user desires, and please the user.
The only sane way to address this is using a control plane. A well-built harness can do this; a sandbox can do this; hell, a carefully-chosen `umask` can do this; but both of those are liable to introduce notification fatigue in the user.
This is akin to politely asking guests to to steal your jewels. If your jewels are in the living room, and your guests have unfettered access to the living room, this technique will only work for the most trustworthy of guests.
It was just a popup: "Hello. This is virus from Albania. Due to poor technology in country, I cannot harm your computer directly. But since you are honest person, please delete some important files from computer and mail this file to at least 3 other people!"
Claude.md is an equally effective defensive tool.
But sorry if you lost some files from reading that.
It's not hard, it's trivial. Most folks here are constantly working with containers. You know how to run a container with a local directory mounted in it.
For myself, I've been using Lima (https://lima-vm.io/) to reduce even that little bit of extra work. Lima works cross-platform leveraging native virtualisation or containerisation, and has some useful capabilities for using agents.
This is a new pattern for me, I'm curious what others are doing.
[0] https://pubs.aeaweb.org/doi/pdfplus/10.1257/0002828067772121...
What? No, but the random 3rd party software you run on your computer, must be limited by you in some way, haven't we learned this even after the AUR, npm and LLM shenanigans we've dealt with for decades at this point?
No, don't ask the model "Please don't go outside this directory", you limit the runtime (via VMs, containers, unix permissions, whatever) so it only has access to what it should, not more.
Same goes for any software, not just agents or chat clients or whatever. Any 3rd party software you don't want to have access to your entire computer, you need to run in this way.
That's scary. Really should run it under different user with carefully assigned permissions.
If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.
https://github.com/ashishb/amazing-sandbox
The real enforcement has to be done via methods that YOU can't easily bypass, or they will bypass (OS-level prohibitions, etc).
And the user even paid $99/month or more for having their data leaked.
You can't gaslight something into not doing something bad. There has to be a hard security control that prevents it doing something bad. And if you don't know what it's capable of because it's non-deterministic then you have to start with a default block everything. This should have never been possible with any sensible design.
On my first point again, ethics and engineering both went out of the window when fast and shiny came along. This is disgraceful.
I swear, people hear the word LLM and their brain resets when it comes to good software practices.
Did VMs suddenly stop existing? Kata containers? An RHEL box with SEL?
It's like there's a new technology and everyone suddenly decided to shutoff their brain when it comes to basic security.
Install with
pi install npm:pi-supergrok
Give the bot it's own machine and only copy to it that which one would want DOGE having access to. Not a virtual machine, the bot will eventually escape. Give it a little RasPi or mini-PC with maximum power savings enabled, disk quotas and bandwidth limits. Instruct it to operate within those limits. Install a custom CA cert and force it's traffic through a Squid SSL Bump MitM proxy so that one can monitor what URL's it goes to and what data it is transferring.
This is probably most important for anyone operating agents on corporate systems. I would suggest corporate security should take interest in this idea so they have a good answer for auditors, insurance companies, investors and customers.
In this way I'm not afraid of letting the agents totally lose on my computer.
But it turns out even in the container, there are footguns that have occasionally made the news by being fired: few projects don't have any external resources and when credentials with any form of write access happen to make it into the container (even if it's just a session cookie) agents might jump at the opportunity.
Note entirely perfect, but will be enough against anyone not actively exploiting kernel privilege escalation bugs.
Instead of generating patches, this exposes the agent's checkout as a git remote though.
Most similar tools (and I believe your tooling as well?) bind-mount the repository checkout from the host into the container. This was always a source of user and permission errors for me, since you have to align the user ids inside and outside the container. Also, some build tools don't like it when the repo is on a different filesystem (bind mount vs container root) than the rest of the system. So I made rumpelpod just bake the repo checkout into the base image that the container is launched from, and since then I haven't had any issues like this.
For giving the agent access to a docker daemon I found sysbox to work very well. Usually the advice for nested containers is to pass in the host's docker socket into the container. But that would break the outer container isolation completely, since access to the docker daemon is equivalent to root access on the host. With sysbox it's trivial to run a nested docker instance inside the outer container. I haven't tried it with podman yet though. In theory sysbox is just another OCI runtime, but there's probably some tinkering required to get it to work.
It brings some extra sandboxing features compared to just a docker container.
For instance it also maps the secrets that the agent harness has access to. And has an allow list for outgoing http connections. And there's a way for agents to request extra access. And once approved by the user from an external cli the policy is updated and hot reloaded.
It's pretty recent. So time will tell how robust it is / will become. What its longevity will be. After all; in a time of LLMs one off experiments are cheap. Longtime nurturing not so much.
Putting it in ALL CAPS!
It's simple sandboxing based solely on unix file permissions. Albeit weak, I find the isolation sufficient. Until I'm shown otherwise it seems like a good compromise given how easy it is.
You can also create iptables rules matching on the user, so this technique is useful for applications where you want to restrict network traffic as well, and don't need stronger or more fine-grained isolation mechanisms.
> Until I'm shown otherwise it seems like a good compromise
Agreed. In my case I went a long way running the Pi harness in a (simple, rootfull) docker container. As the project I worked on relied on a standardized docker compose stack for local dev and testing, I realized I could automate more if only my agent could use docker. Ultimately, the need for docker for my agent grew when testcontainer [2] was introduced in the project. That's when I finally took the time to setup a VM with incus [3], and now I can let the agent go wild with docker inside the VM.
This is at least one example where more isolation is required. Otherwise, the dedicated Linux user, if it works for you, is by far the easiest and most pragmatic solution IMO.
[1] https://docs.docker.com/engine/install/linux-postinstall/#ma... ; https://wiki.debian.org/Docker
[2] https://testcontainers.com/getting-started/
[3] https://linuxcontainers.org/incus/docs/main/
I don't think the LLM had anything to do with this decision at all. It looks like the Grok tool starts a session by deterministically kicking off a full upload of the user's current repository (and maybe their directory if not version tracked? Not clear if this user had previously run "git init" in their home directory) to Grok's servers.
One possible "innocent" explanation could be that xAI then run vector embedding on every file to help later provide the right context. I don't think thats a worthwhile tradeoff here, especially since other popular coding agents get by just using grep/ripgrep run locally.
/s
https://cursor.com/docs/agent/tools/search
So file contents are uploaded for embedding/indexing, but supposedly none of those contents are stored at Cursor after embedding.
Child pornography and sexual pictures uncensored on Grok? Its not Elon Musk issue until it becomes his issue.
Let get Elon Musk, the richest person on the planet with his own propagandaplatform off the hook eh?
Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok Build team to have a toggle from the server to let the program to upload your entire codebase to a Google Cloud Storage bucket. It is not an agent decision, the program is written by the Grok team, can be dissembled and seeing the logic wild-open.
The whole point of LLMs is that you can stop writing rigorous rules in a programming or config language with hard-to-learn syntax, and can resort to natural language instead. You pay for that with the chance for misunderstandings rising to similar levels as in human interaction. That's the tradeoff. Always has been, always will be.
We have a lot of implicit assumptions when it comes to security. If we leave out the step of formalizing these assumptions into exact rules and instead stick to ambiguous and unclear natural language all the time, then these things will happen.
> I knew two guys with sub 2.5 GPAs in college,
Gah! The horror! They should have been sent to the labor camps.
This is a stupid stupid thing to "allow," for every party involved here.
You're a stupid programmer if you're letting these things touch your files.
You're a stupid company if you're letting Grok run wild.
We're a stupid industry if we're not warning everybody about how ridiculous this all is.
Run any cloud-based AI agents in VM/container and map your host's local folders to guest OS as needed.
Takes more effort that default way, I know.
... that's guaranteed vuln-free, right?