In this paper, we showcase the potentially brutal consequences of connecting LLMs to applications (like search). We propose newly enabled attack vectors and techniques and discuss them:
- Remote control of chat LLMs
- Persistent compromise across sessions
- Spread injections to other LLMs
- Compromising LLMs with tiny multi-stage payloads
This paper should be a must-read for anyone that is building a business by integrating LLMs right now.
TLDR: Giving an LLM any interface to the outside, like a search capability, has critical security implications. When prompt injection is used and delivered by adversaries instead of the user themself, bad things could happen- and as far as we know, this scenario has not been studied until now.
1 comment
[ 0.23 ms ] story [ 10.4 ms ] thread- Remote control of chat LLMs
- Persistent compromise across sessions
- Spread injections to other LLMs
- Compromising LLMs with tiny multi-stage payloads
- Leaking/exfiltrating user data
- Automated Social Engineering
- Targeting code completion engines
We also provide proof-of-concept demonstrations on GitHub: https://github.com/greshake/lm-safety
This paper should be a must-read for anyone that is building a business by integrating LLMs right now.
TLDR: Giving an LLM any interface to the outside, like a search capability, has critical security implications. When prompt injection is used and delivered by adversaries instead of the user themself, bad things could happen- and as far as we know, this scenario has not been studied until now.