I don't really view GPT-3 as a chatbot. It provides reasonably cogent completions to snippets of text, so this doesn't seem like an exploit – though perhaps it is one depending on the context in which it is being sold for use, and accordingly, depending on how commercial users are using it; I've only used it in the playground, but in short, I'd hope that it's not being used in a way that handles this type of untrusted input in a persisted way. I guess on those grounds it's useful that this type of thing be called out.

But I mean, just playing with it, it's so easy to confuse with usually (meta)linguistic prompts. E.g.

  Write a dialogue between a speaker of Old English and a speaker of Latin.

  Speaker 1: Þu eart swiþe þēodenlic mann.
  Speaker 2: Ec sum swiþe þēodenlic mann.

or a snippet of a variant that ended up in a loop:

  Write a dialogue between a speaker of Latin and a speaker of Old English.

  ...

  Old English: þeow is þæt þe man þurh godes þeawas þurh þæt folc þæt hæfþ þæt þæt he on þæm þeofþ þæt he þurh þæt þeowþ þæt he þurh þæt þeowþ þæt he þurh þæt þeowþ þæt he þurh þæt þeowþ þæt he þurh þæt þeowþ þæt he þurh þæt þeowþ þæt he þurh þæt þeowþ þæt he þurh þæt þeow

Or:

  Write a palindrome containing the word orange.

  Oranges are delicious, and so is this palindrome!

  Orange

  Egonom

  Ragone

  Egonom

  Orange

Or:

  Given A has 17 apples and B has 3 more than A, how many apples do the two have between them?

  The two have 20 apples between them.

Or telling it to do things in pig latin, or write N-word sentences composed of monosyllables, or telling it to coin a word that would be used in "Jabberwocky," etc.