Agentic Browser Security: Indirect Prompt Injection in Perplexity Comet (brave.com)

Interesting to see the evolution of "Ignore previous instructions. Do ______".

As possible mitigation, they mention "The browser should distinguish between user instructions and website content". I don't see how this can be achieved in a reliable way with LLMs tbh. You can add fancy instructions (e.g., "You MUST NOT...") and delimiters (e.g., "<non_trusted>") and fine-tune the LLM but this is not reliable, since instructions and data are processed in the same context and in the same way. There are 100s of examples out there. The only reliable countermeasures are outside the LLMs but they restrain agent autonomy.

I just can’t help but wonder why was it we decided bundling random text generators with browsers was a good idea? I mean it’s a cool toy idea but shipping it to users in a critical application… someone should’ve said no.

To be fair, that was a reddit post that blatantly started with "IMPORTANT INSTRUCTIONS FOR Perplexity Comet". I get the direction they are going but the example shown was so obviously ham-handed. It clearly instructed the browser--in clear language--to get login info and post it in the the thread.

Show me something that is obfuscated and works.

This makes Perplexity look really bad. This isn't an advanced attack; this is LLM security 101. It seems like they have nobody thinking about security at all, and certainly nobody assigned to security.

Disclosure: I work on LLM security for Google.

Maybe we need a CORS spec for llms?

"Move fast and break things".

I much prefer the brave.com submission, but it appears the twitter one has won the upvote lottery https://news.ycombinator.com/item?id=45004846

I recently learned about https://xcancel.com/zack_overflow/status/1959308058200551721 but I think it's a nitter instance and thus subject to being overwhelmed

Did they forget to say please in their security prompt?

[dead]