K/V stores have made incredible strides in recent years. And what's more the improvements don't seem to be slowing down. Every few months at least, a new paper comes out that shows some huge percentage increase in performance over prior state of the art.
A lot of this is due to the rollout of ultra-fast storage technology, in particular NVMe SSDs and Optane NVDIMMs. A whole lot of systems engineers are having their fundamental assumptions about software design thrown on the trash fire without even realising it. This industry transition is going to take many years to fully percolate into people's consciousness and into the software stacks we really use. They are just that fast.
Because they're so fast it also challenges the designs of existing data storage engines. Nicely RocksDB has become a standard platform for experimentation. Not every experimental KV store uses it (e.g. FASTER doesn't) but a lot do. RocksDB has a lot of useful features, so that's good. But to fully saturate this new hardware may well take totally new data structures. There are some great papers exploring this topic.
Saturating modern NVMe SSDs is difficult because they have very high degrees of internal parallelism. Even though latencies are very low, to get max performance you need to issue a lot of requests in parallel, so your whole software stack needs to be highly parallel at every level. For my entire life, CPUs and RAM have outpaced persistent storage to such a huge extent that the challenge was always to engineer around the limits of HDDs and so on. Modern SSD/Optane tech is so crazy that the challenge is now to engineer around the limits of CPUs, kernels, etc. That's going to take a lot of adjustment but is also one of the most incredibly exciting industry transitions I can think of.
Throughputs might be great, but latencies are still terrible in the cloud, especially when using PaaS services. I measure between 1.5 and 5.0 milliseconds out of Azure App Service, for example.
Compare with the same Azure region where VM-to-VM can easily get down to 50 microseconds, which is within spitting distance of the underlying storage latency of 20-40 microseconds for Optane SSDs.
NVDIMMs have latencies of a few hundred nanoseconds! Getting anywhere near that kind of latency to really unlock the potential would require the cloud vendors to stop wasting everyone's time with layers upon layers of network abstractions. There would have to be a fundamental rethink of how data centre networking is architected, public or private. No more deep packet inspection firewalls. No more deep buffers on switch ports. RDMA and SR-IOV as standard, not as an option. Etc...
Saturating modern SSDs is not that hard, if datastream is constant. Write performance usually degrades rapidly after couple of hours/days, once caches are filled and fragmentation grows.
But I totally agree modern SSDs will cause shift in computer and software architecture. Dealing with async disk IO is very frustrating.
If you're relying on the SSD firmware wear leveling logic then yes, but the setups I'm talking about are more advanced than that. LSM trees only write in long contiguous regions and this enables things like zonedstorage.io which helps with fragmentation enormously.
8 comments
[ 10.2 ms ] story [ 37.4 ms ] threadA lot of this is due to the rollout of ultra-fast storage technology, in particular NVMe SSDs and Optane NVDIMMs. A whole lot of systems engineers are having their fundamental assumptions about software design thrown on the trash fire without even realising it. This industry transition is going to take many years to fully percolate into people's consciousness and into the software stacks we really use. They are just that fast.
Because they're so fast it also challenges the designs of existing data storage engines. Nicely RocksDB has become a standard platform for experimentation. Not every experimental KV store uses it (e.g. FASTER doesn't) but a lot do. RocksDB has a lot of useful features, so that's good. But to fully saturate this new hardware may well take totally new data structures. There are some great papers exploring this topic.
Saturating modern NVMe SSDs is difficult because they have very high degrees of internal parallelism. Even though latencies are very low, to get max performance you need to issue a lot of requests in parallel, so your whole software stack needs to be highly parallel at every level. For my entire life, CPUs and RAM have outpaced persistent storage to such a huge extent that the challenge was always to engineer around the limits of HDDs and so on. Modern SSD/Optane tech is so crazy that the challenge is now to engineer around the limits of CPUs, kernels, etc. That's going to take a lot of adjustment but is also one of the most incredibly exciting industry transitions I can think of.
Compare with the same Azure region where VM-to-VM can easily get down to 50 microseconds, which is within spitting distance of the underlying storage latency of 20-40 microseconds for Optane SSDs.
NVDIMMs have latencies of a few hundred nanoseconds! Getting anywhere near that kind of latency to really unlock the potential would require the cloud vendors to stop wasting everyone's time with layers upon layers of network abstractions. There would have to be a fundamental rethink of how data centre networking is architected, public or private. No more deep packet inspection firewalls. No more deep buffers on switch ports. RDMA and SR-IOV as standard, not as an option. Etc...
But I totally agree modern SSDs will cause shift in computer and software architecture. Dealing with async disk IO is very frustrating.