Transparent compression would also be a useful feature here. On a lot of workloads it would result in the replication cost being more like 1.5x than the 3x you’d typically deploy FDB with, especially if you ran zstd training in the background and made a custom dictionary instead of just using LZ4.
You could also just use a filesystem that does compression on top of it. Might be better? Another thing to watch out for is that if you format it ext4 don't make enormous as it will write lots and lots of superblocks. Found out the hard way.
It's definitely a cool idea. I think it makes sense to make it configurable, since some workloads might write already compresses bytes and then it's not possible to compress them any further.
This looks very interesting. How do you think this concept will perform on larger files (assuming it goes past prototype).
For example, when we started storing data chunks in Cassandra, we found that Cassandra's performance was not optimal due to the way it was dealing with compactions and deleted data.
I'm not familiar enough with FoundationDB's data model design, so would be interesting to know what do you think about possible developments of this project.
Since it is a block device, all files are splitted into chunks of 4 kb (or other value if configured) and from the database perspective there is no difference between large and small files.
For deletions, FoundationDB is able to delete a key range in O(log(n)) time, without using tombstones, and it doesn't use compactions (because it uses B-tree instead of LSM-tree) so I don't think there will be any impact on performance.
Right now TRIM operations are not supported yet, so instead of getting deleted blocks will be marked as "free" by a filesystem and then reused later.
That's helpful, thanks, clarifies key differences in design of Cassandra vs FoundationDB. Are you using a lot of FDB in prod? What are your impressions so far?
My next step is to implement a locking mechanism to prohibit simultaneous mounts and data corruption, and then I'll work on providing a Container Storage Interface to support easy integration with k8s.
I don't think there will be major deviations from the roadmap defined in the readme :)
7 comments
[ 3.1 ms ] story [ 26.1 ms ] threadFor example, when we started storing data chunks in Cassandra, we found that Cassandra's performance was not optimal due to the way it was dealing with compactions and deleted data.
I'm not familiar enough with FoundationDB's data model design, so would be interesting to know what do you think about possible developments of this project.
Since it is a block device, all files are splitted into chunks of 4 kb (or other value if configured) and from the database perspective there is no difference between large and small files.
For deletions, FoundationDB is able to delete a key range in O(log(n)) time, without using tombstones, and it doesn't use compactions (because it uses B-tree instead of LSM-tree) so I don't think there will be any impact on performance.
Right now TRIM operations are not supported yet, so instead of getting deleted blocks will be marked as "free" by a filesystem and then reused later.
What are your next steps with this project?
My next step is to implement a locking mechanism to prohibit simultaneous mounts and data corruption, and then I'll work on providing a Container Storage Interface to support easy integration with k8s.
I don't think there will be major deviations from the roadmap defined in the readme :)