Sooner or later I believe, there will be models which can be deployed locally on your mac and are as good as say Sonnet 4.5. People should shift to completely local at that point. And use sandbox for executing code generated by llm.
Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be sure.
Unlike gemini then you don't have to rely on certain list of whitelisted domains.
> Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be sure.
The problem is that people want the agent to be able to do "research" on the fly.
> Gemini is not supposed to have access to .env files in this scenario (with the default setting ‘Allow Gitignore Access > Off’). However, we show that Gemini bypasses its own setting to get access and subsequently exfiltrate that data.
They pinky promised they won’t use something, and the only reason we learned about it is because they leaked the stuff they shouldn’t even be able to see?
Can we state the obvious of that if you have your environment file within your repo supposed protected by .gitignore you’re automatically doing it wrong?
For cloud credentials you should never have permanent credentials anywhere in any file for any reason best case or worse case have them in your home directory and let the SDK figure out - no you don’t need to explicitly load your credentials ever within your code at least for AWS or GCP.
For anything else, if you aren’t using one of the cloud services where you can store and read your API keys at runtime, at least use something like Vault.
Who would have thought that having access to the whole system can be used to bypass some artificial check.
There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go.
No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the default config.
Fundamentally, with LLMs you can't separate instructions from data, which is the root cause for 99% of vulnerabilities.
Security is hard man, excellent article, thoroughly enjoyed.
I mean, agent coding is essentially copypasting code and shell commands from StackOverflow without reading them. Or installing a random npm package as your dependency.
Should you do that? Maybe not, but people will keep doing that anyway as we've seen in the era of StackOverflow.
There's nothing specific to Gemini and Antigravity here. This is an issue for all agent coding tools with cli access. Personally I'm hesitant to allow mine (I use Cline personally) access to a web search MCP and I tend to give it only relatively trustworthy URLs.
> Personally I'm hesitant to allow mine (I use Cline personally) access to a web search MCP and I tend to give it only relatively trustworthy URLs.
Web search MCPs are generally fine. Whatever is facilitating tool use (whatever program is controlling both the AI model and MCP tool) is the real attack vector.
Copilot will prompt you before accessing untrusted URLs. It seems a crux of the vulnerability that the user didn't need to consent before hitting a url that was effectively an open redirect.
What are "untrusted URLs" ? Or, more to the point: What are trusted URLs?
Prompt injection is just text, right? So if you can input some text and get a site to serve it it you win. There's got to be million of places where someone could do this, including under *.google.com. This seems like a whack-a-mole they are doomed to lose.
Interesting report. Though, I think many of the attack demos cheat a bit, by putting injections more or less directly in the prompt (here via a website at least).
I know it is only one more step, but from a privilege perspective, having the user essentially tell the agent to do what the attackers are saying, is less realistic then let’s say a real drive-by attack, where the user has asked for something completely different.
> Though, I think many of the attack demos cheat a bit, by putting injections more or less directly in the prompt (here via a website at least)
What difference does that make? The prompt is to read a website and the injection is on that website hidden in html. People aren't going to read the HTML of every website before they scrape it, so this is not an unrealistic vulnerability.
Even worse, it ran arbitrary commands to get around its own restrictions. This just confirms if Antigravity tries to scrape a website with user generated content for any reason, whether the user provides the link or not, you have left your entire machine vulnerable.
OCR'ing the page instead of reading the 1 pixel font source would add another layer of mitigation. It should not be possible to send the machine a different set of instructions than a person would see.
I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following:
- A) Process untrustworthy input
- B) Have access to private data
- C) Be able to change external state or communicate externally.
It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any combo of them, imho).
[EDIT] added "or communicate externally" to option C.
It baffles me that we've spent decades building great abstractions to isolate processes with containers and VM's, and we've mostly thrown it out the window with all these AI tools like Cursor, Antigravity, and Claude Code -- at least in their default configurations.
Does anyone else find it concerning how we're just shipping alpha code these days? I know it's really hard to find all bugs internally and you gotta ship, but it seems like we're just outsourcing all bug finding to people, making them vulnerable in the meantime. A "bug" like this seems like one that could have and should have been found internally. I mean it's Google, not some no-name startup. And companies like Microsoft are ready to ship this alpha software into the OS? Doesn't this kinda sound insane?
I mean regardless of how you feel about AI, we can all agree that security is still a concern, right? We can still move fast while not pushing out alpha software. If you're really hyped on AI then aren't you concerned that low hanging fruit risks bringing it all down? People won't even give it a chance if you just show them the shittest version of things
While an LLM will never have security guarantees, it seems like the primary security hole here is:
> However, the default Allowlist provided with Antigravity includes ‘webhook.site’.
It seems like the default Allowlist should be extremely restricted, to only retrieving things from trusted sites that never include any user-generated content, and nothing that could be used to log requests where those logs could be retrieved by users.
And then every other domain needs to be whitelisted by the user when they come up before a request can be made, visually inspecting the contents of the URL. So in this case, a dev would encounter a permissions dialog asking to access 'webhook.site' and see it includes "AWS_SECRET_ACCESS_KEY=..." and go... what the heck? Deny.
Even better, specify things like where secrets are stored, and Antigravity could continuously monitor the LLM's to halt execution if a secret ever appears.
Again, none of this would be a perfect guarantee, but it seems like it would be a lot better?
The agen already bypassed the file reading filter with cat, couldn't it just bypass the URL filter by running wget or a python script or hundreds of other things it has access to through the terminal? You'd have to run it in a VM behind a firewall.
You're telling the agent "implement what it says on <this blog>" and the blog is malicious and exfiltrates data. So Gemini is simply following your instructions.
It is more or less the same as running "npm install <malicious package>" on your own.
Ultimately, AI or not, you are the one responsible for validating dependencies and putting appropriate safeguards in place.
right but this product (agentic AI) is explicitly sold as being able to run on its own. So while I agree that these problems are kind of inherent in AIs... these companies are trying to sell it anyway even though they know that it is going to be a big problem.
We really are only seeing the beginning of the creativity attackers have for this absolutely unmanageable surface area.
I ma hearing again and again by collegues that our jobs are gone, and some are definitely going to go, thankfully I'm in a position to not be too concerned with that aspect but seeing all of this agentic AI and automated deployment and trust that seems to be building in these generative models from a birds eye view is terrifying.
Let alone the potential attack vector of GPU firmware itself given the exponential usage they're seeing. If I was a state well funded actor, I would be going there. Nobody seems to consider it though and so I have to sit back down at parties and be quiet.
I think it depends on where you work. I do quite a lot of work with agentic AI, but it's not like it's much of a risk factor when they have access to nothing. Which they won't have because we haven't even let humans have access to any form of secrets for decades. I'm not sure why people think it's a good idea, or necessary, to let agents run their pipelines, especially if you're storing secrets in envrionment files... I mean, one of the attacks in this article is getting the agent to ignore .gitignore... but what sort of git repository lets you ever push a .env file to begin with? Don't get me wrong, the next attack vector would be renaming the .env file to 2600.md or something but still.
That being said. I think you should actually upscale your party doomsaying. Since the Russian invasion kicked EU into action, we've slowly been replacing all the OT we have with known firmware/hardware vulnerabilities (very quickly for a select few). I fully expect that these are used in conjunction with whatever funsies are being build into various AI models as well as all the other vectors for attacks.
Antigravity was also vulnerable to the classic Markdown image exfiltration bug, which was reported to them a few days ago and flagged as "intended behavior"
I'm hoping they've changed their mind on that but I've not checked to see if they've fixed it yet.
One source of trouble here is that the agent's view of the web page is so different from the human's. We could reduce the incidence of these problems by making them more similar.
Agents often have some DOM-to-markdown tool they use to read web pages. If you use the same tool (via a "reader mode") to view the web page, you'd be assured the thing you're telling the agent to read is the same thing you're reading. Cursor / Antigravity / etc. could have an integrated web browser to support this.
That would make what the human sees closer to what the agent sees. We could also go the other way by having the agent's web browsing tool return web page screenshots instead of DOM / HTML / Markdown.
The prompt injection doesn’t even have to be in 1px font or blending color. The malicious site can just return different content based on the user-agent or other way of detecting the AI agent request.
68 comments
[ 4.2 ms ] story [ 65.7 ms ] threadEdit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be sure. Unlike gemini then you don't have to rely on certain list of whitelisted domains.
The problem is that people want the agent to be able to do "research" on the fly.
They pinky promised they won’t use something, and the only reason we learned about it is because they leaked the stuff they shouldn’t even be able to see?
For cloud credentials you should never have permanent credentials anywhere in any file for any reason best case or worse case have them in your home directory and let the SDK figure out - no you don’t need to explicitly load your credentials ever within your code at least for AWS or GCP.
For anything else, if you aren’t using one of the cloud services where you can store and read your API keys at runtime, at least use something like Vault.
There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go.
No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the default config.
Fundamentally, with LLMs you can't separate instructions from data, which is the root cause for 99% of vulnerabilities.
Security is hard man, excellent article, thoroughly enjoyed.
Should you do that? Maybe not, but people will keep doing that anyway as we've seen in the era of StackOverflow.
Web search MCPs are generally fine. Whatever is facilitating tool use (whatever program is controlling both the AI model and MCP tool) is the real attack vector.
Hope google can do something for preventing prompt injection for AI community.
Prompt injection is just text, right? So if you can input some text and get a site to serve it it you win. There's got to be million of places where someone could do this, including under *.google.com. This seems like a whack-a-mole they are doomed to lose.
I know it is only one more step, but from a privilege perspective, having the user essentially tell the agent to do what the attackers are saying, is less realistic then let’s say a real drive-by attack, where the user has asked for something completely different.
Still, good finding/article of course.
What difference does that make? The prompt is to read a website and the injection is on that website hidden in html. People aren't going to read the HTML of every website before they scrape it, so this is not an unrealistic vulnerability.
Even worse, it ran arbitrary commands to get around its own restrictions. This just confirms if Antigravity tries to scrape a website with user generated content for any reason, whether the user provides the link or not, you have left your entire machine vulnerable.
- A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally.
It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any combo of them, imho).
[EDIT] added "or communicate externally" to option C.
[1] https://simonwillison.net/2025/Nov/2/new-prompt-injection-pa... [2] https://ai.meta.com/blog/practical-ai-agent-security/
I mean regardless of how you feel about AI, we can all agree that security is still a concern, right? We can still move fast while not pushing out alpha software. If you're really hyped on AI then aren't you concerned that low hanging fruit risks bringing it all down? People won't even give it a chance if you just show them the shittest version of things
> However, the default Allowlist provided with Antigravity includes ‘webhook.site’.
It seems like the default Allowlist should be extremely restricted, to only retrieving things from trusted sites that never include any user-generated content, and nothing that could be used to log requests where those logs could be retrieved by users.
And then every other domain needs to be whitelisted by the user when they come up before a request can be made, visually inspecting the contents of the URL. So in this case, a dev would encounter a permissions dialog asking to access 'webhook.site' and see it includes "AWS_SECRET_ACCESS_KEY=..." and go... what the heck? Deny.
Even better, specify things like where secrets are stored, and Antigravity could continuously monitor the LLM's to halt execution if a secret ever appears.
Again, none of this would be a perfect guarantee, but it seems like it would be a lot better?
You're telling the agent "implement what it says on <this blog>" and the blog is malicious and exfiltrates data. So Gemini is simply following your instructions.
It is more or less the same as running "npm install <malicious package>" on your own.
Ultimately, AI or not, you are the one responsible for validating dependencies and putting appropriate safeguards in place.
1. There are countless ways to hide machine-readable content on the blog that doesn't make a visible impact on the page as normally viewed by humans.
2. Even if you somehow verify what the LLM will see, you can't trivially predict how it will respond to what it sees there.
3. In particular, the LLM does not make a proper distinction between things that you told it to do, and things that it reads on the blog.
I ma hearing again and again by collegues that our jobs are gone, and some are definitely going to go, thankfully I'm in a position to not be too concerned with that aspect but seeing all of this agentic AI and automated deployment and trust that seems to be building in these generative models from a birds eye view is terrifying.
Let alone the potential attack vector of GPU firmware itself given the exponential usage they're seeing. If I was a state well funded actor, I would be going there. Nobody seems to consider it though and so I have to sit back down at parties and be quiet.
That being said. I think you should actually upscale your party doomsaying. Since the Russian invasion kicked EU into action, we've slowly been replacing all the OT we have with known firmware/hardware vulnerabilities (very quickly for a select few). I fully expect that these are used in conjunction with whatever funsies are being build into various AI models as well as all the other vectors for attacks.
I'm hoping they've changed their mind on that but I've not checked to see if they've fixed it yet.
https://x.com/p1njc70r/status/1991231714027532526
Agents often have some DOM-to-markdown tool they use to read web pages. If you use the same tool (via a "reader mode") to view the web page, you'd be assured the thing you're telling the agent to read is the same thing you're reading. Cursor / Antigravity / etc. could have an integrated web browser to support this.
That would make what the human sees closer to what the agent sees. We could also go the other way by having the agent's web browsing tool return web page screenshots instead of DOM / HTML / Markdown.
“it’s going to obey rules that are are enforced as conventions but not restrictions”
Which is what you’re doing if you expect it to respect guidelines in a config.
You need to treat it, in some respects, as someone you’re letting have an account on your computer so they can work off of it as well.
likewise for the bad guys