Question: Does FDB have any sort of "find" method in its API? As a key/value store I understand that I can set/get keys, but is there any way to find keys based on nested values?
A database is going to do a query like that in one of two ways: scan over (all of|a column family of) the data, or look in some kind of index on "someproperty".
In our approach this kind of functionality is the responsibility of a higher "layer" of the system above our key/value API. There are lots of ways of organizing and indexing data with different tradeoffs between search and update performance for different kinds of queries. We support pretty much all of them-- not by building any of them into the core, but by providing a general transaction facility that permits arbitrary indices to be updated.
For example: in a database like Mongo, queries like that one don't scale linearly even if there is an index, because you have to ask every shard to look in its local index. In FoundationDB, you would store the index in the database as keys like (someproperty_index, "123", _id) and an index read would only need to go to a single server.
It can be a hassle to create all the higher level functionality yourself, though! Over time, we and others will be offering more "opinionated" layers to save you time.
What a great demo! We're currently building a critical service on MongoDB and feel a bit leery about some of the issues others have expressed at scale. Would be interested in testing FoundationDB for our purpose in the near future.
Thanks! We thought since it was a hit in person we should share it a bit broader.
You mention scale - do know that what you're seeing in the video is probably the weakest FoundationDB cluster ever assembled :) Our in-house testing cluster is a 48 machine, 192 core setup, and it works great and scales almost linearly. You can see our performance numbers at www.foundationdb.com/performance (though those results are outdated now, the tests were on a cluster half the size and several versions ago).
If you'd like to chat about your particular use case, send me an email at nick at our domain.
I'd have to ask one of the guys that put it together for specifics in the morning, but the OS is Ubuntu Linux, and the applications auto launching are of course the FoundationDB server which then connects to the rest of the cluster and begins doing work, and also a client with a Firefox browser running the testing / demo app which increments the counter, changes the circles, and uses (I think) HTML 5 to render the images.
The boxes are by a company called Zotac. They're like $200 without drives and we put cheap consumer SSD's in them. The monitors are USB powered, and driven by USB - they are from a company called DoubleSight.
Funny story - given the amount of people at Disrupt that were not at all interested in databases, we had probably just as many people ask us detailed questions about "those cool little computers" as we did our database :)
Thanks. We haven't published pricing yet but we are aiming to make it in the ballpark of the hardware & operational costs of the machines required to run a particular size cluster. I.e. if you are running a FoundationDB cluster at 50% load then the licensing costs shouldn't be out of sync with the costs of owning and operating, or renting, the machines in the cluster.
It does look likely that we will open source Flow at some point, but it will take a good amount of work to do it properly and right now there is plenty of it to go round. Always good to hear interest for it though!
15 comments
[ 476 ms ] story [ 1352 ms ] threadAlso covered are single and multiple machine hard power failures, rejoining the cluster, and total cluster power loss and recovery, all in one take.
Happy to answer questions.
I.e. fbd.find({someproperty: "123"})...
[0] http://foundationdb.com/documentation/beta1/data-modeling.ht...
In our approach this kind of functionality is the responsibility of a higher "layer" of the system above our key/value API. There are lots of ways of organizing and indexing data with different tradeoffs between search and update performance for different kinds of queries. We support pretty much all of them-- not by building any of them into the core, but by providing a general transaction facility that permits arbitrary indices to be updated.
For example: in a database like Mongo, queries like that one don't scale linearly even if there is an index, because you have to ask every shard to look in its local index. In FoundationDB, you would store the index in the database as keys like (someproperty_index, "123", _id) and an index read would only need to go to a single server.
It can be a hassle to create all the higher level functionality yourself, though! Over time, we and others will be offering more "opinionated" layers to save you time.
You mention scale - do know that what you're seeing in the video is probably the weakest FoundationDB cluster ever assembled :) Our in-house testing cluster is a 48 machine, 192 core setup, and it works great and scales almost linearly. You can see our performance numbers at www.foundationdb.com/performance (though those results are outdated now, the tests were on a cluster half the size and several versions ago).
If you'd like to chat about your particular use case, send me an email at nick at our domain.
Funny story - given the amount of people at Disrupt that were not at all interested in databases, we had probably just as many people ask us detailed questions about "those cool little computers" as we did our database :)
What are the node costs in your licensing model ? Also any plans to release Flow as open source ?
It does look likely that we will open source Flow at some point, but it will take a good amount of work to do it properly and right now there is plenty of it to go round. Always good to hear interest for it though!