Cdb: Add support for cdb64 (cdb.cr.yp.to)

Interesting, never heard of this before. I'm assuming the use case is when your data is too large to conveniently fit into memory?

Haven't there been 64-bit ports of CDB for ages?

Title: cdb: Intro (please use the original title) https://news.ycombinator.com/newsguidelines.html

Can't this be implemented as a PHF: https://en.wikipedia.org/wiki/Perfect_hash_function

CDB is an interesting format, optimized for read-heavy write-rarely[1] random lookups on slow media. This isn’t a very common requirement these days, but it’s convenient for very specific use cases.

[1] You “update” by overwriting the entire file. This is remarkably fast and means that there’s no overhead/tracking for empty space, but it does mean you probably want this to be a fairly rare operation.

I rolled my own cdb reader library for a project a few years ago, and wrote up my notes on the format and its internals here: https://search.feep.dev/blog/post/2022-12-03-cdb-file-format

GALACTIC SCALE QMAIL that can run efficiently on a 486 AND survive a supernova!

I'm kind of surprised I hadn't heard of this, I could see this being something useful for a few projects. Historically for things in this space I've used RocksDB but RocksDB has given me headaches with unpredictable memory usage for large data sets.

Now I’m curious about working around the writer limitations….

cdb is a fun format to implement! highly recommend it.

What's Bernstein working on that would have demanded this?