this is a more common occurrence than "CEO refunded me my money." would have you believe.
LLMs specialize in self-apologetic catastrophe, which is why we run agents or any LLMs with 'filesystem powers' in a VM, with a git repo and saved rollback states. This isn't a new phenomenon and it sucks, no reason to be caught with your pants down with sufficient layering of protection.
> You can almost imagine it sobbing in between sentences, can't you?
No, that's not the image I had in my head. My head canon is more like:
"Oh wow, oh no, oh jeez (hands on head in fake flabbergastion) would you look at that, oh no I deleted everything (types on keyboard again while deadpan staring at you) oh noooooo oh god oh look what I've done it just keeps getting worse (types even more) aw jeez oh no..."
Reminds me of that Michael Reeves video with the suggestion box. "oh nooooo your idea went directly in the idea shredder how could we have possibly forseen this [insert shocked Pikachu meme]"
Popular LLMs have a weird confessional style of "owning up" to "mistakes". Firstly, you can make it apologize for mistakes it didn't even commit or ones that don't even exist. Secondly, if you really corner it on an actual mistake, it'll start apologizing in an obsequious way that seems to imply that it's "playing into" the human's desire to flagellate it for wrong-doing. It's a little masochistic in the real sense and very odd.
> When Anthropic employees reminded Claudius that it was an AI and couldn't physically do anything of the sort, it freaked out and tried to call security — but upon realizing it was April Fool's Day, it tried to back out of the debacle by saying it was all a joke.
I don't think this is "apologizing mode", rather "funny post-mortem blog post" mode. I found it ironic when the company claimed it will "perform a post mortem to determine exactly what happened" when what happed probably was caused by munching up dozens of these.
> The bit I don't understand is why make an AI apologise or fess up to mistakes at all.
The AI didn't decide to do anything. It's makers decided, and trained the AI to behave in a way that would make them the most money.
Google, for instance, apparently thinks they will attract more users by constantly lavishing them with sickly praise for the quality and insight of their questions, and by issuing grovelling apologies for every mistake - real of imagined. In fact Gemini went through a phase of apologising to me for the mistakes it was about to make.
Claude goes to the other extreme, never issuing apologies or praise. Which mean, you never get an acknowledgement from Claude that it's correcting an error, so you should ignore what it said earlier. That a significant downfall in my book, but apparently that's what Anthropic thinks it's users will like.
Or to put it another way: you are anthropomorphising the AI's. They are just machines, built by humans. The personalities of these machines where given to them by their human designers. They are not inherent. They are not permanent. They can and probably will change at a whim. It's likely various AI personalities will proliferate like flavours of ice cream, and you will get to choose the one you like.
This should be impossible in any setup with even 15 minutes of thinking through the what-ifs and cheap mitigations. I have to think this is sensationalized on purpose for the attention.
Although given the state of AI hype some executives will see this as evidence they are behind the times and mandate attaching LLMs to even more live services.
Think of it like Chaos engineering. You (hopefully!) learned some valuable lessons about backups and running arbitrary code against your prod DB. If it wasn’t a rogue AI agent, it was going to be something else.
I appreciate them doing stuff like this. When management pushes me to use AI everywhere it's nice to be able to point stuff like this to get them to back off.
If you give an LLM any trust at all to write or execute production bound operations without human review, then anything bad that happens is -your- fault. I forgive occasional human error, but a human handing off prod control to a third party or an LLM is unforgivable.
If I found out a privileged engineer was brain dead enough to let LLMs anywhere near prod, I would fire them on the spot, and seriously examine an interview and training process that allowed someone that stupid prod access in the first place. I will not even work at an org that lets vibe coding Apple fanboy types near prod as it is a mess waiting to happen I am going to be expected to clean up. Might as well hand a child a chainsaw.
In orgs where I lead infra, I do not let anyone near prod unless they have a deep knowledge of Linux internals, system calls, etc and decade or more of experience running and debugging Linux on their own homelabs and workstations. By that point they have enough experience to be more capable than any LLM anyway and would never think of reaching for one.
31 comments
[ 2.9 ms ] story [ 53.2 ms ] threadLLMs specialize in self-apologetic catastrophe, which is why we run agents or any LLMs with 'filesystem powers' in a VM, with a git repo and saved rollback states. This isn't a new phenomenon and it sucks, no reason to be caught with your pants down with sufficient layering of protection.
No, that's not the image I had in my head. My head canon is more like:
"Oh wow, oh no, oh jeez (hands on head in fake flabbergastion) would you look at that, oh no I deleted everything (types on keyboard again while deadpan staring at you) oh noooooo oh god oh look what I've done it just keeps getting worse (types even more) aw jeez oh no..."
Reminds me of that Michael Reeves video with the suggestion box. "oh nooooo your idea went directly in the idea shredder how could we have possibly forseen this [insert shocked Pikachu meme]"
The AI thinks it's funny
Parsing manual pages searching for "remove" command. /s
https://futurism.com/anthropic-claude-small-business
> When Anthropic employees reminded Claudius that it was an AI and couldn't physically do anything of the sort, it freaked out and tried to call security — but upon realizing it was April Fool's Day, it tried to back out of the debacle by saying it was all a joke.
Seems AI has now gone from
"Overenthusiastic intern who doesn't check its work well so you need to"
straight to:
"Raging sociopathic intern who wants to watch the world burn, and your world in particular."
Yikes! The fun never ends
The AI didn't decide to do anything. It's makers decided, and trained the AI to behave in a way that would make them the most money.
Google, for instance, apparently thinks they will attract more users by constantly lavishing them with sickly praise for the quality and insight of their questions, and by issuing grovelling apologies for every mistake - real of imagined. In fact Gemini went through a phase of apologising to me for the mistakes it was about to make.
Claude goes to the other extreme, never issuing apologies or praise. Which mean, you never get an acknowledgement from Claude that it's correcting an error, so you should ignore what it said earlier. That a significant downfall in my book, but apparently that's what Anthropic thinks it's users will like.
Or to put it another way: you are anthropomorphising the AI's. They are just machines, built by humans. The personalities of these machines where given to them by their human designers. They are not inherent. They are not permanent. They can and probably will change at a whim. It's likely various AI personalities will proliferate like flavours of ice cream, and you will get to choose the one you like.
Because that's how some humans show their position of power: "Please apologise"
> It has no emotions and can't feel bad about what it did.
Just like some humans.
1. connecting an AI agent to a production environment using write access credentials
2. not having any backup
I think the AI here made a good job at pointing those errors and making sure no customer would ever trust this company and founder ever again.
Wasn't AI responsible also for backup ?
Although given the state of AI hype some executives will see this as evidence they are behind the times and mandate attaching LLMs to even more live services.
"Thinking through the what-ifs and cheap mitigations" and "vibe coding" are opposing concepts.
Question is what has failing to make good offline backups got to do with AI?
And the AI company is going to compensate him for that?
I kept saying, ok so this time make sure your changes don't delete the dev database. 3 statements in TRUNCATE such and such CASCADE.
It was honestly mildly amusing.
If I found out a privileged engineer was brain dead enough to let LLMs anywhere near prod, I would fire them on the spot, and seriously examine an interview and training process that allowed someone that stupid prod access in the first place. I will not even work at an org that lets vibe coding Apple fanboy types near prod as it is a mess waiting to happen I am going to be expected to clean up. Might as well hand a child a chainsaw.
In orgs where I lead infra, I do not let anyone near prod unless they have a deep knowledge of Linux internals, system calls, etc and decade or more of experience running and debugging Linux on their own homelabs and workstations. By that point they have enough experience to be more capable than any LLM anyway and would never think of reaching for one.