I assume this is for hashing. I've seen several hashing algorithms turn to hardware AES instructions before, but I haven't seen any evidence that this technique outperforms state-of-the-art hashes like RapidHash (https://github.com/Nicoshev/rapidhash) in either quality or speed.
For any complex system, there's never one single trick or design choice that makes it fast. It's always a large amount of engineering (or exaggerations, of course).
So… it’s not durable? Durable doesn’t mean “survives a process restart”, it means “durably saved to persistent storage”. For example, this “durable” mode wouldn’t survive power loss.
You're right, that mode provides process crash recovery, not power-loss durability. The benchmark compares it against fjall’s equivalent buffered-WAL mode.
Indeed, a common enough pattern for etcd is to run it backed by a RAMdisk and have multi-az availability + periodic backups + tolerance at a business level to be OK losing some recent data.
Word choice matters. Defaults matter. People will go "well it says durable right here" and while arguably, yes, they should RTFM, it would still be great if tool-builders did not set the shotgun's default state to State::AT_FOOT. It would be nice if every paragraph of technical writing that I have to do need not be burdened by a thousand asterisks of "durable in this context means something other than durable".
Agreed. Good design is when the things do what you expect them to do without reading the manual, don't reuse wording with other meaning in the wrong way. That way if you do encounter nee wording, you know you should read the manual.
As of a couple years ago, mmap actually has a MAP_SYNC flag that makes it durable in the DB sense. The caveat is that it requires DAX on the file and so comes with a whole bunch of restrictions w.r.t. filesystem, storage media and even CPU architecture.
Pretty much... paranoid() seems to be the real durable() which isn't a great look for a database project.
Being able to recover a db without corruption beyound losing the last few writes is a pretty useful feature, and buys a lot of performance, but it would be better to label that clearly, as a reasonable expectation on the durable() preset would be for it to be Durable.
Absolutely. If they just dirty some pages in memory and return back to the client the benchmarks will look "insanely fast".
I have nothing against this being a non-default option in a db/kv engine but anything advertising to be durable and not fsyncing by default is something I would stay away from. To me it's like a litmus test of how well the author knows/cares data durability and not destroying users data.
durable() syncs periodically on flush, WAL rotation, and clean close; paranoid() is the sync-before-ack mode. This is clarified in the README, and the benchmarks report all three modes separately. Other KV-stores that you see on the market, do this too. It's a performance tradeoff most applications make. Sync on every write kills every optimization. See the benchmark table for example.
I give a little leeway to distributed systems that replicate and don't flush since there's a bit of middle ground assuming they're in different fault domains. Garage object storage defaults to that
However, this doesn't appear to be the case here...
Unsurprisingly, performance goes to crap when sync is enabled.
Is fsync needed for every write in all projects?
I am willing to trade database speed for the loss of the last written data within a 1-10ms window once or twice a year for non-financial and other non-critical data. So, power loss shouldn't be such a frequent event when there are active transactions at that moment, right?
It’s usually very difficult in a KV db to have an efficient operation that returns the item deleted. You’d need a transaction API to do it reliably. The challenge is concurrent writes are impossible to serialize against without transactions.
The API is likely wrong for an io_uring shaped future. Buffer pool for reads to grab memory is a lot better than each pending read having to provide a buffer for the whole duration of the I/O wait.
The benchmark is setup to test 80 MiB dataset on a machine with 32 GiB RAM, which doesn't represent a typical "database" workload. How does the key-value store perform on larger than RAM datasets? MMAP is incredibly fast when the dataset fits in memory, but it can slow to a crawl when it doesn't, especially if the workload is mostly random point lookups.
More or less. I think the use of CPU specific instructions can make an compiled program different than "the rest". Although nowadays some compilers are clever enough to do better than manual optimization.
Michael Abrash wrote about such optimizations on the late 90's regarding Pentium versus its predecessors, and then everyone started using Python, Ruby, whatever for full stack applications, beyond plain OS scripting tasks.
61 comments
[ 2.2 ms ] story [ 28.4 ms ] thread> TurboKV's persisted Bloom-filter format uses hardware AES.
Also, built-in LZ4 compression.
I would expect SIMD to be used for scans.
> Appended to the WAL without a per-write sync
So… it’s not durable? Durable doesn’t mean “survives a process restart”, it means “durably saved to persistent storage”. For example, this “durable” mode wouldn’t survive power loss.
mmap is nice but it doesn’t support durable semantics in the way that we usually mean with databases.
if a write is acknowledged it should not be forgotten, which is not what this is.
If your system has a reasonable tolerance for power failure (multi-az multi-cloud), this can provide much better throughput
Agreed. Good design is when the things do what you expect them to do without reading the manual, don't reuse wording with other meaning in the wrong way. That way if you do encounter nee wording, you know you should read the manual.
Being able to recover a db without corruption beyound losing the last few writes is a pretty useful feature, and buys a lot of performance, but it would be better to label that clearly, as a reasonable expectation on the durable() preset would be for it to be Durable.
I have nothing against this being a non-default option in a db/kv engine but anything advertising to be durable and not fsyncing by default is something I would stay away from. To me it's like a litmus test of how well the author knows/cares data durability and not destroying users data.
However, this doesn't appear to be the case here...
Unsurprisingly, performance goes to crap when sync is enabled.
This is pretty old now but has some useful fsync/sec numbers which can be completely divorced from other I/O performance https://www.percona.com/blog/fsync-performance-storage-devic...
https://youtu.be/b2F-DItXtZs
https://github.com/ncruces/go-sqlite3
And I'm still having fun.
https://github.com/ncruces/go-sqlite3/pull/421
https://labs.tomasino.org/gnu-recutils/
Sharding would be so easy! Just run "split" and spread the chunks over a bunch of servers, then run "recutils" with "parallel" to retrieve!
Truly webscale.
This DB doesn’t have a transaction API.
It seems pretty ubiquitous yet not a part of the core rust libs.
Insanely fast was making 8 bit games possible at all.
If not, please just link the repo and its own title, no need for hype.