> we had a significant issue in ChatGPT due to a bug in an open source library, for which a fix has now been released and we have just finished validating.
WHICH open source library? Seems like they fucked up and are looking for a scapegoat. There's no excuses for issues like this. Also, I believe this is yet again one more reason for companies and developers to consider row-level security [0], it simply erases the possibility of a malformed query returning data that shouldn't be accessible altogether.
The bug was discovered in the Redis client open-source library, redis-py. As soon as we identified the bug, we reached out to the Redis maintainers with a patch to resolve the issue. Here’s how the bug worked:
- We use Redis to cache user information in our server so we don’t need to check our database for every request.
- We use Redis Cluster to distribute this load over multiple Redis instances.
- We use the redis-py library to interface with Redis from our Python server, which runs with Asyncio.
- The library maintains a shared pool of connections between the server and the cluster, and recycles a connection to be used for another request once done.
- When using Asyncio, requests and responses with redis-py behave as two queues: the caller pushes a request onto the incoming queue, and will pop a response from the outgoing queue, and then return the connection to the pool.
- If a request is canceled after the request is pushed onto the incoming queue, but before the response popped from the outgoing queue, we see our bug: the connection thus becomes corrupted and the next response that’s dequeued for an unrelated request can receive data left behind in the connection.
- In most cases, this results in an unrecoverable server error, and the user will have to try their request again.
- But in some cases the corrupted data happens to match the data type the requester was expecting, and so what gets returned from the cache appears valid, even if it belongs to another user.
At 1 a.m. Pacific time on Monday, March 20, we inadvertently introduced a change to our server that caused a spike in Redis request cancellations. This created a small probability for each connection to return bad data.
This bug only appeared in the Asyncio redis-py client for Redis Cluster, and has now been fixed.
A few questions this still doesn't answer: why was there credit card data in their Redis cluster, and why were they the only ones affected that I know of? Still, this response seems much more reasonable than what I was expecting from OpenAI considering the seriousness of the issue. Anyways, thank you very much for the summary, I didn't find this blog post in the publications that were talking about the issue.
A think a reasonable lesson from this is to double check that the returned data belongs to the user you're about to serve it to. So include some way to associate the owning user with each cache entry, so it can be verified later when it's recalled.
Wouldn’t this require creating a database role for every user? Then your backend needs to be able to assume every user role, which still seems to allow for bugs.
This seems to be a common misconception. You do not need a role for every user. You can assign session variables, at least in Postgres, and use them in your policies.
> Also, I believe this is yet again one more reason for companies and developers to consider row-level security [0], it simply erases the possibility of a malformed query returning data that shouldn't be accessible altogether.
That requires 1:1 DB user to real user mapping + one connection to DB per users, it is entirely horrible performance-wise, which is why it is so rarely used for anything more than separating apps from eachother.
Maybe they should be reminded that redis-py is under the MIT license.
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"Why didn't you simply have a test for an unexpected failure case which arose as a result of moving pieces across a distributed system?" - whateveracct
It is pretty outrageous, yes. If Alice logs in and I correctly request Alice's details from the database, it's reasonable for me to expect that the database either returns Alice's details or an error. It shouldn't be up to application-layer programmers to code checks to make sure that a database doesn't return the wrong data 1% of the time. That kind of test is a test for the developers of Asyncio and Redis to implement, not users of Asyncio and Redis.
I mean I have tested such things many times over the years. I'm no stranger to Redis cache bugs. Being a professional means expecting more of yourself.
Also they literally say in the postmortem that the same bug was resulted in exceptions most of the time and only sometimes leaked data. So all those exceptions were right there as a giant clue.
It's telling of character that Sam doesn't take responsibility, and blames it on an open source package in literally the same breath.
OpenAI is a company that we are going to have a significant relationship with into the future, so it's worth understanding the characteristics of the leadership of that company.
A real leader would simply have said "we fixed it, sorry", instead of "we fixed it, but it was someone else's fault".
It reminds me of those "I deleted the production database and got fired" stories ..... I always think it's management to blame if it is possible to delete the production database, but management like to blame someone further down the tree.
“The guy” controls a pipeline that did not sanitize its inputs.
The DB equivalent is letting a drop table statement run against prod.
It’s where supply chain attacks occur.
Full transparency would mean opening their code, CI/CD pipelines; all their tech stack would be public. So yes, that is what I want. Not this South Park “we are so sorry.”
This all really should be openly available and accountable to public scrutiny if we’re going to call it the future of the internet.
The only indication we need to know about the ethics of OpenAI is the fact that it's a private for-profit company that claims the name "OpenAI", where "open" has very clear meaning in its sphere of relevance (technology, computing, and software).
This feels like a bad faith take. It's reasonable to say _why_ the thing occurred, that's not blaming. "We fixed it, the root cause was X" is a perfectly fine thing to say.
I don't think this is a strong reason to infer something about Altman's character per se. I think more concerning is that the "Actions we've taken" portion of their blogpost about the incident are all relatively narrowly tied to _this specific issue_, and the "Where do we go from here" section just says that Redis is really valuable to them.
They do _not_ say, "here's a list of broad changes to our practices which will allow us discover similar issues before they reach production and touch customer data".
On the one hand I feel empathy towards the engineers because I know third party bugs can feel somewhat unfair and obscure.
On the other hand if I understand correctly, the bug arises from an edge case from an error that was very common. I constantly receive these errors using the chat interface, and that’s an organization issue. There should be an SRE team or someone to notice them and try to fix them.
The issue was relatively minor in the grand scheme of things, but the "it was a bug in an open source library, so not our fault" attitude from the CEO, especially when addressing a user data leak, is concerning.
Instead of blame deflection, the communication users (and companies) want to hear is – "it was our fault, and we are overhauling our security and vendor audit process to make sure it doesn't happen again."
It sounds exactly like the Silicon Valley excuse we all know and love.
Sorry your account's been shut down, it must have been the silly algorithm/silly source code/silly engineers in the workshop again!
When you're making a tool that could quite likely change the course of how people use and interact with computers, these excuses don't fly.
Hope it doesn't happen again, but the old mantra of "move fast, break things, apologize later" seems to apply here just like with every other SV company.
>the old mantra of "move fast, break things, apologize later" seems to apply here just like with every other SV company.
I mean, obviously? They would not be able to ship things at this speed with so few employees if they were being conservative. No value judgement, just saying we should not be surprised.
50 comments
[ 1.8 ms ] story [ 111 ms ] thread[0]: https://www.postgresql.org/docs/current/ddl-rowsecurity.html
The bug was discovered in the Redis client open-source library, redis-py. As soon as we identified the bug, we reached out to the Redis maintainers with a patch to resolve the issue. Here’s how the bug worked:
- We use Redis to cache user information in our server so we don’t need to check our database for every request.
- We use Redis Cluster to distribute this load over multiple Redis instances.
- We use the redis-py library to interface with Redis from our Python server, which runs with Asyncio.
- The library maintains a shared pool of connections between the server and the cluster, and recycles a connection to be used for another request once done.
- When using Asyncio, requests and responses with redis-py behave as two queues: the caller pushes a request onto the incoming queue, and will pop a response from the outgoing queue, and then return the connection to the pool.
- If a request is canceled after the request is pushed onto the incoming queue, but before the response popped from the outgoing queue, we see our bug: the connection thus becomes corrupted and the next response that’s dequeued for an unrelated request can receive data left behind in the connection.
- In most cases, this results in an unrecoverable server error, and the user will have to try their request again.
- But in some cases the corrupted data happens to match the data type the requester was expecting, and so what gets returned from the cache appears valid, even if it belongs to another user.
At 1 a.m. Pacific time on Monday, March 20, we inadvertently introduced a change to our server that caused a spike in Redis request cancellations. This created a small probability for each connection to return bad data.
This bug only appeared in the Asyncio redis-py client for Redis Cluster, and has now been fixed.
[0] https://openai.com/blog/march-20-chatgpt-outage
That requires 1:1 DB user to real user mapping + one connection to DB per users, it is entirely horrible performance-wise, which is why it is so rarely used for anything more than separating apps from eachother.
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
OpenAI is a company that we are going to have a significant relationship with into the future, so it's worth understanding the characteristics of the leadership of that company.
A real leader would simply have said "we fixed it, sorry", instead of "we fixed it, but it was someone else's fault".
It reminds me of those "I deleted the production database and got fired" stories ..... I always think it's management to blame if it is possible to delete the production database, but management like to blame someone further down the tree.
Do you want full transparency or what? Jesus.
The DB equivalent is letting a drop table statement run against prod.
It’s where supply chain attacks occur.
Full transparency would mean opening their code, CI/CD pipelines; all their tech stack would be public. So yes, that is what I want. Not this South Park “we are so sorry.”
This all really should be openly available and accountable to public scrutiny if we’re going to call it the future of the internet.
I think it's better than just saying "we fixed it, sorry"
They do _not_ say, "here's a list of broad changes to our practices which will allow us discover similar issues before they reach production and touch customer data".
On the other hand if I understand correctly, the bug arises from an edge case from an error that was very common. I constantly receive these errors using the chat interface, and that’s an organization issue. There should be an SRE team or someone to notice them and try to fix them.
Instead of blame deflection, the communication users (and companies) want to hear is – "it was our fault, and we are overhauling our security and vendor audit process to make sure it doesn't happen again."
Sorry your account's been shut down, it must have been the silly algorithm/silly source code/silly engineers in the workshop again!
When you're making a tool that could quite likely change the course of how people use and interact with computers, these excuses don't fly.
Hope it doesn't happen again, but the old mantra of "move fast, break things, apologize later" seems to apply here just like with every other SV company.
I mean, obviously? They would not be able to ship things at this speed with so few employees if they were being conservative. No value judgement, just saying we should not be surprised.
(You can't do this here, though, and I've banned the account.)