I don't really understand the continuing appeal for CockroachDB. Yes there is some cool tech, but given that all transactions are synchronously distributed, there's a hard cap on how fast it can be. I have yet to see any significant case studies about real workloads, while their competitors like Vitess are seeing whole-scale adoption from their users - including Slack and HubSpot.
I'm not sure I understand this. Vitess performs synchronous replication to at least two nodes for all write IIUC. The point that I think you're making is that Vitess generally does that in a single region so, for a given write, you see single-region latency. Cockroach can do this too. In fact, in very similar terms. Cockroach allows data to be partitioned and then partitions can be configured with placement constraints, including which regions to place the data. At the end of the day, for queries which touch the equivalent of one shard, you can configure cockroach to experience a latency profile comparable to Vitess.
Additionally, cockroach offers the ability to survive regional failures, and, likely most importantly, it allows you to perform arbitrary, serializable cross-shard transactions. Vitess has some sort of 2PL transaction thing but last time I heard Sugu talk about it, he said something like, nobody uses it.
Since I'm not a Cockroach expert, I'm sure that there are ways to configure it that eliminate some of those latency issues that were common early on. Both Vitess & Cockroach can replicate across regions and survive regional failures.
As for cross-shard transactions, that's mainly what I'm referring to about slowness. Vitess does have 2PC, but as you mention, almost nobody uses it in production. Companies using Vitess architect their data and sharding patterns so that cross-shard transactions aren't necessary. There are definitely cases where that is part of the requirement, but in far fewer cases than many people think.
At the end of the day, I'm still waiting to see any major tech company push serious workloads onto Cockroach. Until that happens at scale, and as I continue to see more and more of those major companies adopting Vitess, it's going to be hard to convince me that Cockroach solves all the problems that they claim to. What they are fantastic at is great docs and beautiful interfaces, both of which are lacking on the Vitess side. :)
> At the end of the day, I'm still waiting to see any major tech company push serious workloads onto Cockroach. Until that happens at scale, and as I continue to see more and more of those major companies adopting Vitess, it's going to be hard to convince me that Cockroach solves all the problems that they claim to. What they are fantastic at is great docs and beautiful interfaces, both of which are lacking on the Vitess side. :)
Fair enough, keep your ears open and you'll hear more. Vitess's distribution model is simpler, less tightly coupled and in that way more easily scalable. It indeed has, at this point, been deployed at scales cockroach has not yet reached.
Does your main complaint boil down to the ability of Vitess to do async cross region replication? If so, sure, crdb will have higher commit latency. This use case comes up but it's not common. The general solution when that matters is to leverage nearby regions.
> As for cross-shard transactions, that's mainly what I'm referring to about slowness.
This has gotten better, but it doesn't get around the speed of light. We've got some ideas we're playing with to offer an increasingly flexible models which expose better tradeoffs here.
> Companies using Vitess architect their data and sharding patterns so that cross-shard transactions aren't necessary. There are definitely cases where that is part of the requirement, but in far fewer cases than many people think.
I think this is really the point of contention. Vitess enables applications that require thousands of database machines to operate at the cost of careful application architecture decision making. If you're running thousands of database VMs you're definitely spending a lot on engineering. Those companies exist, sure, but it's not most of the Fortune 500. I do definitely agree that something like Vitess is superior to the Cassandra-like NoSQL database I used to work with. You can build correct applications with those (and something like Kafka), but it's hard.
Planning out your application and know how to shard it up-front is hard too. Vitess is amazing for people using MySQL and then realizing they need to scale, but then you do need to go change things and drop the transactions.
Anyway, I think on the whole, you're saying very fair things. Vitess leverages a mature database in a nice way. If you've got the scale where you need 10s of thousands of cores, it's a better than any NoSQL database out there. It just does have up-front costs to the application developer.
CRDB is selling the gospel of the distributed transaction. The pitch is that once you get used to the application development power of having it, you won't want to go back. The argument is that the tradeoffs that it offers to provide that are reasonable for global applications that need to operate at the scales of thousands of cores for the database which is more than most OLTP business applications need. Both the scale and set of tradeoffs will get better over time.
There is definitely a magical feeling to spinning up Cockroach on a single binary and just letting it do things for you, vs the much steeper learning curve of Vitess. If you aren't looking for peak performance and are optimizing for rapid iteration and ease of use, that's a more compelling pitch than I've heard before. I've always recommended people choose either Postgres or MySQL, whichever they feel more comfortable running, since they likely won't have to ever migrate off the single db, and they'll have full compatibility for queries, apps, etc.
> Those companies exist, sure, but it's not most of the Fortune 500
That's a very interesting observation on database scale and not one that I've heard before, but I guess makes counter-intuitive sense. Most of the Fortune 500 aren't modern tech companies, and thus may not be as reliant on databases in the same way as a newer company.
> thus may not be as reliant on databases in the same way as a newer company.
I think my perspective on this is a tad different. The world is changing and has changed because of computing. At this point, on some level, every big business needs to effectively leverage technology to compete. The difference, I think, between the infrastructure needs of digital information companies which operate on those massive data scales vs other business has a lot to do with the timescales of notable events that need to be processed. If you want to track every keystroke or page load of every user where users number in the hundreds of millions, sure, that's the sort of problem where getting the data layout finely tuned is valuable. If your problem is tracking sensors on thousands of trains or, even surprisingly, every credit card transaction or retail banking operation, the scales are a lot smaller. How many times a day do you use your bank account vs send a slack message or receive a notification? My guess is that it's a few orders of magnitude off. For what it's worth, there's a reason companies like Oracle and IBM have been making a boat load of money for a very long time. Those databases don't scale horizontally, but most businesses aren't growing in the rate of data creation that they are doing in a way that needs it. The advent of the cloud is slowly gaining traction in industries which have long had large IT budgets going to products people on this forum tend not to talk about because, in part, they aren't easy technologies to access if you aren't in an enterprise setting.
All this being said, there are "new" companies doing real things that are global and mission critical but are building services where the request rate is on the order of human action rather than computer action.
It's not that it can't be, but it is a completely different codebase from Spanner without the luxury of TrueTime and complete control over datacenters that it runs in. Cockroach simply hasn't been pushed to the same levels of performance that Spanner or Vitess are already serving, and likely a few orders of magnitude less. I look forward to seeing case studies when larger enterprises do take it that far.
8 comments
[ 4.5 ms ] story [ 33.1 ms ] threadAdditionally, cockroach offers the ability to survive regional failures, and, likely most importantly, it allows you to perform arbitrary, serializable cross-shard transactions. Vitess has some sort of 2PL transaction thing but last time I heard Sugu talk about it, he said something like, nobody uses it.
As for cross-shard transactions, that's mainly what I'm referring to about slowness. Vitess does have 2PC, but as you mention, almost nobody uses it in production. Companies using Vitess architect their data and sharding patterns so that cross-shard transactions aren't necessary. There are definitely cases where that is part of the requirement, but in far fewer cases than many people think.
At the end of the day, I'm still waiting to see any major tech company push serious workloads onto Cockroach. Until that happens at scale, and as I continue to see more and more of those major companies adopting Vitess, it's going to be hard to convince me that Cockroach solves all the problems that they claim to. What they are fantastic at is great docs and beautiful interfaces, both of which are lacking on the Vitess side. :)
Fair enough, keep your ears open and you'll hear more. Vitess's distribution model is simpler, less tightly coupled and in that way more easily scalable. It indeed has, at this point, been deployed at scales cockroach has not yet reached.
Does your main complaint boil down to the ability of Vitess to do async cross region replication? If so, sure, crdb will have higher commit latency. This use case comes up but it's not common. The general solution when that matters is to leverage nearby regions.
> As for cross-shard transactions, that's mainly what I'm referring to about slowness.
This has gotten better, but it doesn't get around the speed of light. We've got some ideas we're playing with to offer an increasingly flexible models which expose better tradeoffs here.
> Companies using Vitess architect their data and sharding patterns so that cross-shard transactions aren't necessary. There are definitely cases where that is part of the requirement, but in far fewer cases than many people think.
I think this is really the point of contention. Vitess enables applications that require thousands of database machines to operate at the cost of careful application architecture decision making. If you're running thousands of database VMs you're definitely spending a lot on engineering. Those companies exist, sure, but it's not most of the Fortune 500. I do definitely agree that something like Vitess is superior to the Cassandra-like NoSQL database I used to work with. You can build correct applications with those (and something like Kafka), but it's hard.
Planning out your application and know how to shard it up-front is hard too. Vitess is amazing for people using MySQL and then realizing they need to scale, but then you do need to go change things and drop the transactions.
Anyway, I think on the whole, you're saying very fair things. Vitess leverages a mature database in a nice way. If you've got the scale where you need 10s of thousands of cores, it's a better than any NoSQL database out there. It just does have up-front costs to the application developer.
CRDB is selling the gospel of the distributed transaction. The pitch is that once you get used to the application development power of having it, you won't want to go back. The argument is that the tradeoffs that it offers to provide that are reasonable for global applications that need to operate at the scales of thousands of cores for the database which is more than most OLTP business applications need. Both the scale and set of tradeoffs will get better over time.
> Those companies exist, sure, but it's not most of the Fortune 500
That's a very interesting observation on database scale and not one that I've heard before, but I guess makes counter-intuitive sense. Most of the Fortune 500 aren't modern tech companies, and thus may not be as reliant on databases in the same way as a newer company.
I think my perspective on this is a tad different. The world is changing and has changed because of computing. At this point, on some level, every big business needs to effectively leverage technology to compete. The difference, I think, between the infrastructure needs of digital information companies which operate on those massive data scales vs other business has a lot to do with the timescales of notable events that need to be processed. If you want to track every keystroke or page load of every user where users number in the hundreds of millions, sure, that's the sort of problem where getting the data layout finely tuned is valuable. If your problem is tracking sensors on thousands of trains or, even surprisingly, every credit card transaction or retail banking operation, the scales are a lot smaller. How many times a day do you use your bank account vs send a slack message or receive a notification? My guess is that it's a few orders of magnitude off. For what it's worth, there's a reason companies like Oracle and IBM have been making a boat load of money for a very long time. Those databases don't scale horizontally, but most businesses aren't growing in the rate of data creation that they are doing in a way that needs it. The advent of the cloud is slowly gaining traction in industries which have long had large IT budgets going to products people on this forum tend not to talk about because, in part, they aren't easy technologies to access if you aren't in an enterprise setting.
All this being said, there are "new" companies doing real things that are global and mission critical but are building services where the request rate is on the order of human action rather than computer action.