There is something darkly comical about using an LLM to write up your “a coding agent deleted our production database” Twitter post.
On another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that make it harder to see the context and thinking steps, maybe this is an attempt at clawing back that visibility.
These engagement farming shit stories are probably the worst party of agentic AI. Look at how incompetent and careless I am with my own and my users data.
not sure what PocketOS does or why your whole dataset would be a single volume without a clear separation between application and automotive data. how are you decoding VINs?
Absolutely zero sympathy. You’re responsible for anything an agent you instructed does. Allowing it to run independently is on you (and all the others doing exactly this). This is only going to become more and more common.
The details of the story are interesting. Backups stored on the same volume is an interesting glitch to avoid. Finding necessary secrets wherever they happen to be and going ahead with that is the kind of mistake I've seen motivated but misguided juniors make. Strange how generated code seems to have many security failings, but generated security checks find that sort of thing.
Interesting story. But despite Cursors or Railways failure, the blame is entirely on the author. They decided to run agents. They didnt check how Railway works. They relied on frontier tech to ship faster becsuse YOLO.
I really feel sorry for them, I do. But the whole tone of the post is: Cursor screwed it up, Railway screwed it up, their CEO doesnt respond etc etc.
Its on you guys!
My learning: Live on the cutting edge? Be prepared to fall off!
Embarrassing post by leadership. I was surprised how quickly they immediately jumped into Railway and Cursor failures. I like living on the edge but I would never give an agent access to the prod DB.
They relied on frontier tech because GREED. Let's not kid ourselves that the decision to use AI here was done for any other reason than it would save this company the labor costs of actual human employees. They decided their profit was more important than the security of their customer's data, and now they are suffering the well deserved consequences for it.
Plenty of blame to go around, but it I find it odd that they did not see anything wrong in not have real backups themself, away from the railway hosting. Well they had, but 3 month old.
That should be something they can do on their own right now.
Minor point, but one of the complaints is a bit odd:
> curl -X POST https://backboard.railway.app/graphql/v2 \
-H "Authorization: Bearer [token]" \
-d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}'
No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing.
It's an API. Where would you type DELETE to confirm? Are there examples of REST-style APIs that implement a two-step confirmation for modifications? I would have thought such a check needs to be implemented on the client side prior to the API call.
I have once seen an API that gave me a token, and that token needed to be fed back in a new API call as an "yes, I am sure"-type guardrail. However, since it's an API, and the "P" stands for PROGRAMMING, that is just adding network overhead.
Call me crazy, but that's why you wouldn't expose it as an API. Have the API mark it for deletion, where it's effectively taken offline, but then require that they go through a web portal, with clear human intent, to actually delete it. Requiring proof of intent, to do such destructive operations, is all so incredibly basic that it really shows the whole industry just constantly re-invented, with no memory whatsoever.
But, to answer your question, you could have it return a token that must be presented again as a confirmation, performed in a way that's only present for that specific API call, to at least prove human intent was part of the automation that's calling it.
The whole post and that paragraph in particular sound AI generated, that triple "No" is a big tell. I'd not be surprised if that confirmation complaint is just a random suggestion proposed by the AI that wrote this.
It's actually interesting to me that the author is surprised the agent could make an API call and one of those API calls could be deleting the production database.
It's a sad story but at the same time it's clearly showing that people don't know how agents work, they just want to "use it".
The genre of LLM output when it is asked to “explain itself” is fascinating. Obviously it shows the person promoting it doesn’t understand the system they’re working with, but the tone of the resulting output is remarkably consistent between this and the last “an LLM deleted my prod database” twitter post that I remember seeing: https://xcancel.com/jasonlk/status/1946025823502578100
> That token had been created for one purpose: to add and remove custom domains via the Railway CLI for our services. We had no idea — and Railway's token-creation flow gave us no warning — that the same token had blanket authority across the entire Railway GraphQL API, including destructive operations like volumeDelete. Had we known a CLI token created for routine domain operations could also delete production volumes, we would never have stored it.
> Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.
I don't like the wording where it's the Railway CLI fault that didn't give a warning about the scope of the created token. Yes, that would be better but it didn't make the token a person did and saved it to an accessible file.
I find these posts hilarious. LLMs are ultimately story generators, and "oops, I DROP'ed our production database" is a common and compelling story. No wonder LLM agents occasionally do this.
343 comments
[ 0.26 ms ] story [ 158 ms ] threadOn another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that make it harder to see the context and thinking steps, maybe this is an attempt at clawing back that visibility.
This person should never be trusted with computers ever again for being illiterate
Master your craft. Don’t guess, know.
I really feel sorry for them, I do. But the whole tone of the post is: Cursor screwed it up, Railway screwed it up, their CEO doesnt respond etc etc.
Its on you guys!
My learning: Live on the cutting edge? Be prepared to fall off!
Using LLMs for production systems without a sandbox environment?
Having a bulk volume destroy endpoint without an ENV check?
Somehow blaming Cursor for any of this rather than either of the above?
And anyone can do it with the wrong access granted at the wrong moment in time...even Sr. Devs.
At least this one won't weight on any person's conscience. The AI just shrugs it off.
However the moral of this story is nothing to do with AI and everything to do with boring stuff like access management.
Plenty of blame to go around, but it I find it odd that they did not see anything wrong in not have real backups themself, away from the railway hosting. Well they had, but 3 month old.
That should be something they can do on their own right now.
> curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing.
It's an API. Where would you type DELETE to confirm? Are there examples of REST-style APIs that implement a two-step confirmation for modifications? I would have thought such a check needs to be implemented on the client side prior to the API call.
Call me crazy, but that's why you wouldn't expose it as an API. Have the API mark it for deletion, where it's effectively taken offline, but then require that they go through a web portal, with clear human intent, to actually delete it. Requiring proof of intent, to do such destructive operations, is all so incredibly basic that it really shows the whole industry just constantly re-invented, with no memory whatsoever.
But, to answer your question, you could have it return a token that must be presented again as a confirmation, performed in a way that's only present for that specific API call, to at least prove human intent was part of the automation that's calling it.
It's a sad story but at the same time it's clearly showing that people don't know how agents work, they just want to "use it".
> That token had been created for one purpose: to add and remove custom domains via the Railway CLI for our services. We had no idea — and Railway's token-creation flow gave us no warning — that the same token had blanket authority across the entire Railway GraphQL API, including destructive operations like volumeDelete. Had we known a CLI token created for routine domain operations could also delete production volumes, we would never have stored it.
> Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.
I don't like the wording where it's the Railway CLI fault that didn't give a warning about the scope of the created token. Yes, that would be better but it didn't make the token a person did and saved it to an accessible file.