Show HN: Cachegrand – a fast OSS Key-Value store built for modern hardware (github.com)
It scales up really nicely thanks to a year of research and development of the hashtable implemented in cachegrand, on the hardware used for benchmarking, an AMD EPYC 7502P, it was able to reach up to 5mln GET QPS and 4.5mln SET QPS, with batching up to 60mln GET QPS and up to 26MLN SET QPS.
cachegrand is fast, it's fully Open Source, it's under a BSD 3-clause license - it can be used easily as standalone platform or incorporated in other ones without any licensing issue - and we are working to expand the Redis functionalities supported and to impelement a tiered storage to cache more data than the available memory. Longer term our goal is to expand the support to different platforms (e.g. memcache, kafka, etc.), add support to webassembly to have user defined functions and server side events, and of course a network bypass (combining XDP and a lockless FreeBSD tcp/ip stack) and a storage bypass.
Although it can easily used via docker, here a direct link to the latest release https://github.com/danielealbano/cachegrand/releases/tag/v0....
Currently we are focused on supporting Redis, here the list of commands currently implemented https://github.com/danielealbano/cachegrand/blob/main/docs/a...
58 comments
[ 3.1 ms ] story [ 114 ms ] threadMy goal is to make it usable enough from the 0.3.0 where I want to have enough Redis commands supported (hashsets, sets, lists, multiple dbs, hyperloglog, etc.), have more memory control and have an MVP of the on-disk database.
Implementing the data replication, which will come right after, will be fairly easy, thanks to all the logics already built in for the on-disk database, so that should be implemented fairly quickly.
Also I have recently bought 2 x AMD EPYC 7551 so I will setup a better performance regression tracking and a proper benchmark suite to track everything (a contributor is working on leveraging the tests and use them as base for the two things mentioned above)
- https://aerospike.com/blog/aerospike-4-strong-consistency-an...
- https://jepsen.io/analyses/aerospike-3-99-0-3
https://www.linkedin.com/posts/danielesalvatorealbano_cacheg...
In general I am pretty sure cachegrand would be faster than memcache easily but I will add it to the todo list as it makes sense.
Thanks for pointing it out!
Dragonflydb also break Redis license big time.
Since the comment says "Trust me as a committer it smokes aerospike and dragonflydb." we can rule out aerospike and dragonflydb, which leaves us with "it" being Cachegrand. Ain't that deep.
That didn't make sense to me so I assumed they must not be a contributor to cachegrand.
The timeseriesdb is "half" a consequence of having a Write-Ahead-Log that is split in chunks and it's chained. I am saying "half" because the other one is the future addition of secondary index to make it possible and easier to query the internal db properly.
I know that we are talking mainly about Redis right now, but that's just the tip of the iceberg and my long term vision is to build a much more complete and flexible platform which can easily handle streams (e.g. via a Kafka interface) and/or allow you to run more evolved data processing via WASM (e.g. I want yo make possible to calculate rolling window averages in a time-sensible fashion :)).
Is the time series replicated and merged between nodes or does each have its own log for the keys it manages?
The timeseries will depend on the keys, as it's the historical sequence of values, so it will be replicated and merged on the nodes that own that specific key in active-active replication with the required replication mode.
Although not mandatory, for the active-active replication mode cachegrand will provide a front-end proxy that "understands" which is the correct node for a key and send the necessary data to always the same node (or subset of nodes, depending on the configuration).
The replication itself will be last-write-wins, if the order of the writes matter it will be important to write always to the same node (very common pattern to reduce syncing and locking on the replication side).
(Obviously it does happen, but statistically speaking it's almost never)
In terms of "just performance", Redis can easily chew 200k GET RPS on an average low-core count VM, even if an application does 10 Redis queries per request in average it would still take 20k requests to saturate it, if we leave 15% of marging for peak traffic / issues / surprises / etc, it would still take an application handling 17.5k RPS which is an HUGE amount if we think that this would require easily between 50 and 100 machines beefy machines!
I think the biggest limitation nowadays is instead the cost of using "only" memory for the cache and having to use a bunch of different systems to process your data.
Try to imagine what you would be able to do if cachegrand would ingest your stream as kafka-compatible server, run your webassembly compiled script and/or run your ML/AI models (leveraging webassembly) and then let push data to other databases / systems and/or let you access your processed data via the Redis / Memcache / GraphQL interface!
And on top of this, imagine that all these modules (Kafka, Redis, Memcache, GraphQL, etc.) can leverage a network bypass and a nvme bypass to perform super fast I/O.
It's a lot of stuff, but that's my long term goal / vision.
Of course to achieve all of this, you need a blazing fast and very flexible base! We are currently focusing on the Redis support because needs many different bits and pieces and would allows us to have people starting to use cachegrand which is a key to understand if the grand plan makes sense :)
> 17.5k RPS which is an HUGE amount if we think that this would require easily between 50 and 100 machines beefy machines!
Maybe we have different definitions of beefy, but in terms of HTTP, we serve 2-4x this on less than half that.
Thanks!
You said:
Redis can easily chew 200k GET RPS on an average low-core count VM, even if an application does 10 Redis queries per request in average it would still take 20k requests to saturate it... which is an HUGE amount if we think that this would require easily between 50 and 100 machines beefy machines!
Which says you estimate 50-100 machines to saturate one "low-core count VM" Redis. But now you say you meant 50-100 machines for Redis servers?
We run about 20 machines, perhaps the equivalent of 10 "beefy machines", to handle a ~50k/sec request load (with substantially higher peaks). We have less than 100 servers total, most of which are doing asynchronous data processing and not directly in the request pipeline. Our data storage architecture is not really comparable to redis in terms of request load as it's insert/upsert-dominated but the total size is 5-10TB.
> Which says you estimate 50-100 machines to saturate one "low-core count VM" Redis. But now you say you meant 50-100 machines for Redis servers?
No, I was referring to machines running business logic functionalities using Redis as part of their processing pipeline.
Said that, the reason for which I was asking for some numbers was to figure out if my expectation of "very often you need more room for the data than better performance" was making sense.
Because cachegrand will be able to store data on disk and to handle stream writes as well allowing to leverage the time series db, once fully implemented, it will able to cover a number of different needs. This combined with the ability to run Webassembly means that you will also be able to run whatever you want directly in place and data will not have to go out and in from multiple systems to be processing making the pipelines much faster and cheaper to run.
Of course it doesn't take 1 day, especially because I am working on cachegrand during my free time (e.g. At night or over the weekends).
https://news.ycombinator.com/item?id=31563641
TL;DR Unlikely to be apples to apples comparison and you can get much more performance out of Redis easily.
But honestly, performances aren't the "only" goal and they aren't the main goal neither although to get where we want to be of course we need extremely good numbers! :)
Compare with e.g. dragonflydb: https://github.com/dragonflydb/dragonfly/blob/main/docs/api_...
Interesting to see how this will develop over time.
Other commands are planned to be added soon but the ones already implemented cover already a lot of different use cases.
0: https://github.com/dragonflydb/dragonfly/blob/main/LICENSE.m...
There is a reason for which the BSL is not a recognized open source license.
The change date can be changed at any time, the license allows it, almost all the projects I saw using the BSL actually use a sentence like "X years from the release", on top of this it can be even removed and if that would be the case than the paragraph below would apply, which clearly refers to 5 years from the released versions.
> Effective on the Change Date, or the fifth anniversary of the first publicly available distribution of a specific version of the Licensed Work under this License, whichever comes first, the Licensor hereby grants you rights under the terms of the Change License, and the rights granted in the paragraph above terminate.
5 years old software is a dead software :)
The only reason for adopting the BSL is to prevent companies like Amazon to take ownership and don't give back to the project (e.g. elastic), which is totally understandable.
Do you think that future releases, much more feature-complete and flexible, will be given under the apache license as they are meanwhile less feature complete versions aren't? Looking at the facts, I would say the answer is very likely no.
Would you build something in 5 years based on dragonflydb 0.7.0 when most likely there will be a v2.0 or even a v3.0 out there?
When it comes to non open source licenses and companies it should be clear that the companies want (and need) to take care of their own interests: the BSL "change date" is a way to be able to say "yes but it will become OSS" so you can keep using and contributing, like the sop you give to a kid to stop him from crying.
I do agree that 5 year old software is dead software, which is a pretty aggressive number for Dragonfly to take here, but at least it's not 10+ years.
> The only reason for adopting the BSL is to prevent companies like Amazon to take ownership and don't give back to the project (e.g. elastic), which is totally understandable.
I also agree with this. It's a dog-eats-dog-world and businesses have to figure out how to compete.
> The change date can be changed at any time, the license allows it, almost all the projects I saw using the BSL actually use a sentence like "X years from the release", on top of this it can be even removed and if that would be the case than the paragraph below would apply, which clearly refers to 5 years from the released versions.
There is still Git involved here. The authors can't re-write history. A commit from today will convert at the same date regardless of future commits, even if the BSL clause is updated in future commits. Dragonfly pushing a commit publicly seems like it should constitute a "release date" to me!
> When it comes to non open source licenses and companies it should be clear that the companies want (and need) to take care of their own interests: the BSL "change date" is a way to be able to say "yes but it will become OSS" so you can keep using and contributing, like the sop you give to a kid to stop him from crying.
That's the glass-half-empty perspective of this. The glass-half-full perspective is: At least it is going to be _eventually_ Open Source! (which is better than yet-another-proprietary-vendor popping up.)
> Would you build something in 5 years based on dragonflydb 0.7.0 when most likely there will be a v2.0 or even a v3.0 out there?
Almost certainly not. I would be worried about the security vulnerabilities and not having patches to fix them.
But, given that their BSL clause does allow commercial use for a private company or an Open Source project, I would think about if using the latest version was the right tool for the job. I just won't expect to go to AWS and get a managed version of the service -- I'd have to go to Dragonfly (which is their goal).
I am empathetic to both sides of this and I'm happy to see licenses like BSL becoming a thing because I think it will continue to accelerate the rate of innovation. (I've had to debug too many vendor binaries in my life. Even just "source available" is valuable when you have production infrastructure heavily reliant on a piece of software because you can still debug it + fix it without requiring the vendor!)
Let me start saying that no one prevents a company from selling the software with licenses allowing to access the source code, companies even provide the code under the GPL because the BSD or the GPL nowhere state that the code has to be provided for free, big companies have provided the code under open source licenses behind fees for quite a while so far (redhat is one of the examples).
The BSL certainly allows you to access the code without having a license but I would dare to say that most of the companies out there, especially for critical components like this, prefer managed and supported solutions, for which rights to access the source code can be granted.
But in general it's absolutely true that it allows accessing the source code without having to buy the license and this is certainly a plus for the final user.
But that's it :)
I will not say that I know the legal language but I will say that the legal language really differs form what we perceive and a judge or a court perceive and the interpretation is dictated by the national law.
The sentence
> Effective on the Change Date, or the fifth anniversary of the first publicly available distribution of a specific version of the Licensed Work under this License, whichever comes first, the Licensor hereby grants you rights under the terms of the Change License, and the rights granted in the paragraph above terminate.
Use the word "version" which is broad and generic, can be read as "the commit makes a new version" or "the intent for the developer to release a new version".
Do, again, no certainty really ok the long term.
Going open source require sacrifices, I have given for free more than 1 year of research and development I did to build the technology behind the hashtable implemented in cachegrand and I did because I believe that true technological advancement should be available to anyone and meanwhile my work is not a game changer it my small bit to try to help.
That's why when I see a license like BSL pointed out as open source or "quasi" open source I do feel the need to give some context.
Try to imagine if Linus at the time would have decided to give Linux under BSL... Nowadays would be very different!
In general is pretty stable, I let it run for hours under memtier_benchmark hammering it, although I relocated in March and all my stuff is still packed in a warehouse (it's impossible to find an apartment in Zurich, Switzerland) but I am getting a new 2 x AMD EPYC 7551 with 512GB of ram to have plenty of room for testing and benchmarking with local hardware instead of asking favours :)
The multithreading helps to scale the access to the hashtable vertically almost linearly.
The use of the fibers make everything easier because cachegrand doesn't have multiple threads running on the same core fighting for the same resources.
On top of this (and thanks to not having multiple threads fighting for the same resources on the same core), the hashtable uses user-space spinlocks (that are "fake" spinlocks as they can't prevent the kernel to preemept, although there are ways to do it) and the "contention" is spread all across the hashtable. The spinlocks though are only used to change the hashtable (with UPSERT, DEL and Read-Modify-Write operations), they are not required to read from it.
Let's say that thread A, B and C, want to access key1, key2 and key3.
If you an hashtable of 10000 buckets, cachegrand will basically split it in chunks of 14 elements and each chunk will have its own localized lock, so there will be about 714 different possible locks. In general, unless you are writing always to the same key, the writes will be distribuited across all the hashtable hitting different locks so it will not be necessary to wait.
Of course I considered different approches a few years ago when I was researching the hashtable including sharding it across multiple threads and let a thread own a slice of it but decided not to go down this way because instead of having 714 locks, like the case above, you will need to use queues to pass messages between the threads (a bit like Scala does) and "focus" the contention on limited number of memory areas (e.g. if you have 4 threads you will have 4 queues and focus the contention on these 4 queues which is bad). Of course you can use atomic operations to push and pop elements from these queues but it will not be able to scale up as the contention will make this system tremble under heavy load.
cachegrand, thanks to the localized spinlocks, can hit 60 million GET and 27 million SET per second on hardware used for benchmarking.
For the hashtable part you can jump to minute 8.40.
This year I will present my efforts at the P99Conf, the content will be similar.
Most of the performance exploits for the hashtable (which is the driving engine here) are explained after minute 8.30 (although I would suggest you to watch the entire presentstion).
I will present my work at the p99conf as well.
For folks wanting to learn more, we'll have Daniele's session for cachegrand at P99 CONF next month (free, virtual event): https://www.p99conf.io/
[edit: just realized we'll also have Roman Gershman of DragonflyDB. So you can hear two different ways to tackle the issue.]
I also promise to document more the actual inner workings of the various components, I have some scattered documentation here and there which will help me in the task!