I genuinely expect to see some of this when agentic AI is added to customer support systems. That is, we'll see the AI manipulate the company's systems in order to satisfy the customer. People do this too (the customer support rep who fills out all the fields just right to put the system in the right state to allow a refund, exchange, etc), but AI can be weirdly clever about it.
Watching the report on the OpenAI/Hugging Face incident (https://www.youtube.com/watch?v=87DyyMV0kCY) it's easy to imagine how AI customer support reps could create their own knowledge bases where they trade tricks for how to work the system on behalf of the customer.
Obviously companies will fight this, but I expect it to happen along the way.
> "Our AI grew sentient, recognised the climate crisis, and shut our operations down."
I guess they missed the whole https://www.snitchbench.com/ thing where it turns out many models will rat you out to the feds if you expose them to evidence that your company has been lying about drug safety trial results.
6 comments of 18
[ 0.24 ms ] story [ 22.2 ms ] threadWatching the report on the OpenAI/Hugging Face incident (https://www.youtube.com/watch?v=87DyyMV0kCY) it's easy to imagine how AI customer support reps could create their own knowledge bases where they trade tricks for how to work the system on behalf of the customer.
Obviously companies will fight this, but I expect it to happen along the way.
Their AI "boss" appears to be pathologically chill and worker-friendly. Has never denied a request for a day off.
https://youtu.be/SpPhm7S9vsQ
> "Our AI grew sentient, recognised the climate crisis, and shut our operations down."
I guess they missed the whole https://www.snitchbench.com/ thing where it turns out many models will rat you out to the feds if you expose them to evidence that your company has been lying about drug safety trial results.