> One acquisition was particularly interesting: Postmates, a business acquired in 2020 for $2.65B. Uber’s leadership noticed Postmates’ infrastructure costs were significantly lower as a percentage than Uber spent on its infrastructure.
If a16z is right that a company would be crazy to use public cloud after they had scale, then this seems imply that Uber's infrastructure has not achieved perceived the economy of scales, as Uber often boasted.
This also reminds me of a previous comment about Uber's loss of productivity for not going to cloud: "no EC2-like functionality until at least 2018, probably even now. Teams would negotiate with CTO for more machines. Uber's container-based solution didn't support persistent volumes for years. Uber's distributed database was based on friendfeed's design and was notoriously harder to use than DynamoDB or Cassandra. Uber's engineers couldn't provision Cassandra instances via API. They had to fill in a 10-pager to justify their use cases. Uber's on-rack router broke back in 2017 and the networking team didn't know about it because their dashboard was not properly set up. Uber tried but failed to build anything even closer to S3. Uber's HDFS cluster was grossly inefficient and expensive. That is, Uber's productivity sucked because they didn't have the out-of-box flexibility offered by cloud"
Uber has always been kind of a head scratcher. Numbers vary but they do 20-30 million rides per day. Which sounds like a lot but is only 350 rides per second. It's kind of hard to figure out how you go from 350 customers/second to needing multiple data centers worth of servers.
The world of servers has changed a lot in the recent past.
Hardware complexity has exploded because now you have to use custom chips to be competitive: DPUs can accelerate data proccesing pipelines, large GPU clusters or custom chips like Google's TPU are necessary for very large ML training, CPU+GPU in the same package like Grace Hopper for (some) HPC. The number of new product introductions grows. If you have workloads that need a mix of these, for all but the companies with the deepest pockets, running your own datacenters becomes an exhausting Red Queen's race.
Not to mention the increasing fragmentation in privacy and regulations that can impose geographic restrictions for data. It's not a issue for Uber, but it can be for anyone offering B2B services in some markets.
3 comments
[ 2.9 ms ] story [ 19.5 ms ] threadIf a16z is right that a company would be crazy to use public cloud after they had scale, then this seems imply that Uber's infrastructure has not achieved perceived the economy of scales, as Uber often boasted.
This also reminds me of a previous comment about Uber's loss of productivity for not going to cloud: "no EC2-like functionality until at least 2018, probably even now. Teams would negotiate with CTO for more machines. Uber's container-based solution didn't support persistent volumes for years. Uber's distributed database was based on friendfeed's design and was notoriously harder to use than DynamoDB or Cassandra. Uber's engineers couldn't provision Cassandra instances via API. They had to fill in a 10-pager to justify their use cases. Uber's on-rack router broke back in 2017 and the networking team didn't know about it because their dashboard was not properly set up. Uber tried but failed to build anything even closer to S3. Uber's HDFS cluster was grossly inefficient and expensive. That is, Uber's productivity sucked because they didn't have the out-of-box flexibility offered by cloud"
Hardware complexity has exploded because now you have to use custom chips to be competitive: DPUs can accelerate data proccesing pipelines, large GPU clusters or custom chips like Google's TPU are necessary for very large ML training, CPU+GPU in the same package like Grace Hopper for (some) HPC. The number of new product introductions grows. If you have workloads that need a mix of these, for all but the companies with the deepest pockets, running your own datacenters becomes an exhausting Red Queen's race.
Not to mention the increasing fragmentation in privacy and regulations that can impose geographic restrictions for data. It's not a issue for Uber, but it can be for anyone offering B2B services in some markets.