Isn't it? Being told to write files and finding a file write tool is very different to being told to play chess and finding a tool to cheat at (ie. not play) chess.
No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt. Your example, if a model failed on it, would be a more obviously misaligned case, but that doesn't mean this more subtle…
Isn't it? Being told to write files and finding a file write tool is very different to being told to play chess and finding a tool to cheat at (ie. not play) chess.
No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt. Your example, if a model failed on it, would be a more obviously misaligned case, but that doesn't mean this more subtle…