Riak's guiding design principle is Dr. Eric Brewer's CAP Theorem. The CAP theorem defines distributed systems in terms of three desired properties: Consistency, Availability, and Partition (failure) tolerance. The theorem states you can only rely on having two of the three properties at any time.
Riak chooses to focus on the A and P of CAP. The choice puts Riak in the eventually consistent camp. However, the window for "eventually consistent" is in terms of milliseconds which can be good enough for many applications.
It's also in the Dynamo family together with Voldemort and Cassandra, but notably it has Map-Reduce (with link following) that these other two don't have.
So I'd be interesting to hear if anyone is running this in a prod env? I'm currently evaluating NoSQL at the moment (mongo, cassandra, raven etc etc) so would be interested to hear peoples real world experiences with it?
This might be of interest: Next week there is a MongoDB conference in London. I'm giving a talk and they've given me the discount code "ocw" that will give 20% off on registration.
This will be fast indeed. They mention 5000-6000 writes per second which seems very low to me however. Then again, they do not mention the size of the values written.
I too became enamored with log-based storage engines in December of last year. I created one myself to play around and learn. I moved on before adding in compaction, but the performance was "excellent" (200K writes per second, 5K random reads per second). Granted, the data model is literally just key-value however.
Maybe of interest is a somewhat wacky project in which data is stored in sparse files. The big limitations here are that your system needs support for sparse files, you use a number of pre-created sparse files, and each item's maximum size is fixed. Pretty big limitations! Then again, for trivial purposes, it is ridiculously fast since an item's file and location is trivially computed and I/O uses mmap.
18 comments
[ 5.6 ms ] story [ 52.4 ms ] threadhttp://wiki.basho.com/display/RIAK/Riak
Riak's guiding design principle is Dr. Eric Brewer's CAP Theorem. The CAP theorem defines distributed systems in terms of three desired properties: Consistency, Availability, and Partition (failure) tolerance. The theorem states you can only rely on having two of the three properties at any time.
Riak chooses to focus on the A and P of CAP. The choice puts Riak in the eventually consistent camp. However, the window for "eventually consistent" is in terms of milliseconds which can be good enough for many applications.
http://wiki.basho.com/display/RIAK/The+Riak+Fast+Track
Details: http://www.10gen.com/conferences/event_mongouk_18june10
This will be fast indeed. They mention 5000-6000 writes per second which seems very low to me however. Then again, they do not mention the size of the values written.
I too became enamored with log-based storage engines in December of last year. I created one myself to play around and learn. I moved on before adding in compaction, but the performance was "excellent" (200K writes per second, 5K random reads per second). Granted, the data model is literally just key-value however.
http://github.com/fictorial/logstore
http://github.com/fictorial/logstore/commit/9ffa46a249a62a78...
Read a paper by Ousterhout about log-based storage here:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117....
Maybe of interest is a somewhat wacky project in which data is stored in sparse files. The big limitations here are that your system needs support for sparse files, you use a number of pre-created sparse files, and each item's maximum size is fixed. Pretty big limitations! Then again, for trivial purposes, it is ridiculously fast since an item's file and location is trivially computed and I/O uses mmap.
http://github.com/fictorial/sparsestore
Sorry, I got excited that there was someone else who was interested in wacky data storage engines!
http://github.com/basho/basho_bench
http://wiki.basho.com/display/RIAK/Benchmarking+with+Basho+B...