I'm giving the first talk of the session and while I'll be talking about ideas we've actually built and tested in Convex, the point I want to get across is that I think the industry has gone in the wrong direction with…
Yes the readsets aren't based on lists of objects but rather the data ranges that a query touches, i.e., they don't suffer from phantom read anomalies. If you were to attempt to fetch all items from an empty shopping…
Convex is FSL licensed which means you can basically do whatever you want with it re running a service, other than directly competing with Convex, i.e., reselling the Convex platform. There are no limitations on you…
(convex cofounder here) Convex is in a very comfortable financial position but it is also open source: https://github.com/get-convex/convex-backend
Convex cofounder here. Yep the database transaction logic, timestamp assignment, concurrency control, etc is all custom. This is a pretty key requirement in our ability to perform dynamic subscriptions and automatic…
Convex cofounder here. Initial focus has been low-latency OLTP database workloads, which is pretty tied to the user-facing end-to-end reactivity angle we're pushing. As a bonus feature we also have built-in strongly…
(kmod was the founding engineer on the project and the guy who came up with the name!)
We should also clarify that Sujay and I are part of the old old team :) The current team have been doing an awesome job since then! (a bunch of the early MP folks work at Convex now)
"It’s fairly easy to design a system with astronomically high durability numbers. 24 nines is a mean time to failure of 1,000,000,000,000,000,000,000,000 years. When your MTTF dwarfs the age of the universe then it…
We had a lot of requests for open sourcing MP but I don't think it would have been very useful. The code itself is interesting to look at but the system is designed for very large data sets, has a lot of moving parts to…
Yeah we have a few talks about this and a chapter about this very issue in https://www.oreilly.com/library/view/seeking-sre/97814919788.... Totally agree that in a well designed system the sources of data loss are…
We designed and built the system almost completely ground-up, all the way down to the disk scheduler, but yeah we probably took the term OSD from Ceph.
Yep that’s me! I have my full name on here so wasn’t hiding but good point to be clear about disclosing.
Firebase has been a huge influence but with the rise of serverless there are a new generation of platforms people should be checking out instead, such as Convex, Supabase, etc. Convex in particular is designed far more…
We track the readset for any active subscription. When a new write transaction commits we compare the writeset for the transaction against any active subscriptions. If there's an intersection then it's likely that query…
Thank you! We have a really talented team working with us but we're very actively growing. If anyone is looking for a job in San Francisco: https://www.convex.dev/jobs
This is very nice feedback. The short answer is that we want to solve problems that matter for the people who need them solved.
Thanks for the feedback. We think React is the right community to target right now but the end state for Convex is certainly not limited to React. The specific concern with regards to multiple round trips is scenarios…
I'm not sure what your definition of smart polling is but there is no polling going on here - the query only reruns when the data dependencies change server-side due to a subsequent mutation. Another key distinction is…
Obviously depends on the kind of projects you're working on but in our experience we've been more productive in Rust compared to when we've built large systems in Python, Go, C and C++. This is with regards to long-term…
I've been using Valora for a while now and even though I discovered it through the crypto world, I use it because it solves an actual problem for me. It's been a great solution for venmo-style payments with friends in…
Thanks for the feedback! Glad folks are reading this stuff :) > Would you be able to give more details as to how you use disks as raw block devices, bypassing the filesystem? For example, the kind of abstraction layers…
Yup we do this, and yes, the verifier has been broken before (during early development, not at production scale). We do a number of things here, like taking down a sufficient number of storage nodes in a single region…
For sure. I hope we're all agreeing here :) We've very strong believers that an effective replication strategy is just table stakes and that from there the real risks to durability are the "black swan" events that are…
From what I hear from other major storage providers Magic Pocket is definitely very high up on the paranoia/verification scale. There is of course a cost to all this verification traffic. Any sensible storage provider…
I'm giving the first talk of the session and while I'll be talking about ideas we've actually built and tested in Convex, the point I want to get across is that I think the industry has gone in the wrong direction with…
Yes the readsets aren't based on lists of objects but rather the data ranges that a query touches, i.e., they don't suffer from phantom read anomalies. If you were to attempt to fetch all items from an empty shopping…
Convex is FSL licensed which means you can basically do whatever you want with it re running a service, other than directly competing with Convex, i.e., reselling the Convex platform. There are no limitations on you…
(convex cofounder here) Convex is in a very comfortable financial position but it is also open source: https://github.com/get-convex/convex-backend
Convex cofounder here. Yep the database transaction logic, timestamp assignment, concurrency control, etc is all custom. This is a pretty key requirement in our ability to perform dynamic subscriptions and automatic…
Convex cofounder here. Initial focus has been low-latency OLTP database workloads, which is pretty tied to the user-facing end-to-end reactivity angle we're pushing. As a bonus feature we also have built-in strongly…
(kmod was the founding engineer on the project and the guy who came up with the name!)
We should also clarify that Sujay and I are part of the old old team :) The current team have been doing an awesome job since then! (a bunch of the early MP folks work at Convex now)
"It’s fairly easy to design a system with astronomically high durability numbers. 24 nines is a mean time to failure of 1,000,000,000,000,000,000,000,000 years. When your MTTF dwarfs the age of the universe then it…
We had a lot of requests for open sourcing MP but I don't think it would have been very useful. The code itself is interesting to look at but the system is designed for very large data sets, has a lot of moving parts to…
Yeah we have a few talks about this and a chapter about this very issue in https://www.oreilly.com/library/view/seeking-sre/97814919788.... Totally agree that in a well designed system the sources of data loss are…
We designed and built the system almost completely ground-up, all the way down to the disk scheduler, but yeah we probably took the term OSD from Ceph.
Yep that’s me! I have my full name on here so wasn’t hiding but good point to be clear about disclosing.
Firebase has been a huge influence but with the rise of serverless there are a new generation of platforms people should be checking out instead, such as Convex, Supabase, etc. Convex in particular is designed far more…
We track the readset for any active subscription. When a new write transaction commits we compare the writeset for the transaction against any active subscriptions. If there's an intersection then it's likely that query…
Thank you! We have a really talented team working with us but we're very actively growing. If anyone is looking for a job in San Francisco: https://www.convex.dev/jobs
This is very nice feedback. The short answer is that we want to solve problems that matter for the people who need them solved.
Thanks for the feedback. We think React is the right community to target right now but the end state for Convex is certainly not limited to React. The specific concern with regards to multiple round trips is scenarios…
I'm not sure what your definition of smart polling is but there is no polling going on here - the query only reruns when the data dependencies change server-side due to a subsequent mutation. Another key distinction is…
Obviously depends on the kind of projects you're working on but in our experience we've been more productive in Rust compared to when we've built large systems in Python, Go, C and C++. This is with regards to long-term…
I've been using Valora for a while now and even though I discovered it through the crypto world, I use it because it solves an actual problem for me. It's been a great solution for venmo-style payments with friends in…
Thanks for the feedback! Glad folks are reading this stuff :) > Would you be able to give more details as to how you use disks as raw block devices, bypassing the filesystem? For example, the kind of abstraction layers…
Yup we do this, and yes, the verifier has been broken before (during early development, not at production scale). We do a number of things here, like taking down a sufficient number of storage nodes in a single region…
For sure. I hope we're all agreeing here :) We've very strong believers that an effective replication strategy is just table stakes and that from there the real risks to durability are the "black swan" events that are…
From what I hear from other major storage providers Magic Pocket is definitely very high up on the paranoia/verification scale. There is of course a cost to all this verification traffic. Any sensible storage provider…