Basically I think this is pointing to mongo for replacing traditional website databases and couch for only special cases. Do you guys think this is fair? I haven't spent too much time with either but I'm a little bit suspicious because this is from mongodb.
Anybody know of any other direct comparisons between these?
I don't consider redis durable for similar reasons as mongo - yet redis doesn't hide what it is or what it lacks in terms of durability... so it doesn't bother me that it isn't.
but mongo often downplays it's lack of durability, which makes me wary of using it or recommending it because people don't know how to use it.
This is exactly the problem with MongoDB. It's practical use is a semi-durable, in-memory data store, but they advertise it as a data store of record that can be used for large, >RAM sets of any conceivable use case.
It seemed a bit dishonest seeing benchmarks that showed how MongoDB beats CouchDB in write-performance tests when there was no acknowledgement of write requests comming back to the client.
Not saying that MongoDB wouldn't be faster with full commits turn on, it is just that their design decision to pick that as the default and calling their product a 'database' seemed very strange at best, and dishonest at worst.
For the record: we (MongoDB) have a policy of not posting official Benchmarks. Could you clarify what benchmarks you are referring to that were posted and "dishonest"?
The only benchmarks we have ever done have been for internal testing comparisons between different versions.
Previously there were several third party benchmarks on this page, which we recently took down to stand firmer on our benchmark policy. In the interest of full disclosure however, you may examine the page history to verify that we have always clarified these as unaffiliated third party sourced.
I'm a little biased, being a pretty heavy CouchDB user, but it seems like Mongo is claiming that their auto-sharding is the defining reason to use Mongo. The forthcoming release of Couchbase 2.0 is going to have the same type of functionality, with buckets distributed across the cluster rather than required replication of the full dataset. Couchbase 3.0 should (if I remember correctly) have the functionality for both the key/value store and for view results.
I'm new to both (and noSQL in general), first heard about CouchDB a week or two ago. Would Couch be a good fit for building a logging system, or is there something else I should look into?
I wouldn't use it for a write-heavy logging system ( and I am a big fan of it).
Well actually it depends on what type of logging. If you plan on pouring megabytes of data/sec second into it continuously you might want to find something else. I don't think CouchDB would be a good candidate for that. It might be if you plan on archiving your post-processed logged data for a long term storage but you have to be smart about its format.
Yeah 'logging' can mean a lot of things that it is hard to give a better advice without more information.
So, both MongoDB and Redis are DB's optimized for keeping stuff heavily in memory. I wouldn't use either for heavy logging.
RRDTool is nice, but only for data where you don't mind losing records.
For straight logging, SQL is actually a better solution than a lot of those options. If you need weird queries, something like Elastic Search or Solr on top of your SQL DB can buy you quite a bit of freedom. As can denormalizing your SQL.
One other thing to consider of course, is something like syslog.
HBase might be a good solution, too, but I don't know a whole lot about it. CouchDB would work fine but you'd want to add some operational automation and a proxy to roll over between temporary databases since deleting old logs from a single database won't fully reduce its size and, as stated above, compaction isn't terribly fun or fast but you can avoid it by just flat out deleting old log databases.
Absolutely it would be great for logging. Assuming that you don't need atomicity for your logs (eg, I want to see that log line reflect in my reports _right_ _now_) Couch is a perfect way to dump a huge amount of data really quickly. For reporting, the map/reduce system of Couch is a great fit. There's an initial learning curve -- as with anything, really -- but you'll find that Couch would be a great way to get logged data (which can be pretty unstructured) and report on it easily and quickly.
no, it wouldn't. It would be a horrible fit, imo.
You don't need versioning info for logging, and the extra compacting of the database is another aspect, since with logging heavy applications you will write all the time...
I love CouchDB but i definitely wouldn't use it for logging.
You don't need to worry about the versioning for a document that you never update. You can just blast the logfiles into the database and not worry about conflicts. Similarly, database compaction is really the biggest concern when you're updating documents, not just dumping new ones into the database. I would expect a very linear growth for a logging app.
ok, my initial thought was one document per logfile, containing an increasing amount of loglines, but that's probably the worst approach possible. One document per logline which is appended to the database should be much better.
The problem doesn't come with the inserts, it comes with the deletes, where CouchDB will keep some metadata and stubs around to support its replication use cases. In a pure logging setup this is perhaps less than ideal.
However, a document update in CouchDB is fully ACID (including the A of Atomicity). People often confuse transactions around multiple changes with ACID for reasons that I cannot fathom. CouchDB does not have support the former (they don't scale well in clusters) but absolutely supports the latter.
Couch's map-reduce is quite slow for ad-hoc stuff. It's excellent for frequently viewed data (as you can get stale views, and new views are sort of incrementally done), but it's slow to set up.
I'm not an expert on CouchDB (and I would love someone to correct where my approach is wrong).
I could see having a CouchDB cluster per co-location, recording a document per logging event. Then back haul through replication to a central service for analysis/dashboard use.
The only tricky part I see is to delete old logging data requires a compaction period and according to http://wiki.apache.org/couchdb/Compaction it's entirely possible for the compaction process to be slower then write data and never actually finish.
Edit: Also the back end server doing analysis may need to be a BigCouch cluster to handle the load (depending on size) https://github.com/cloudant/bigcouch
No. CouchDB is good for storing important data, slow at writing, and bad at ad-hoc querying.
MongoDB is lightning fast at writing (provided you haven't got indexes, or your indexes fit in RAM), good at ad-hoc stuff, and a little more durable than /dev/null. Just remember to back up occasionally.
I don't see how these posts are relevant for a general purpose comparison. It is one use case which doesn't fit CouchDB. Such articles give users a wrong impression. Fact is that MongoDB doesn't fit this use case either. Good solutions handle the following graph separate from the main implementation using specialized graph databases.
I am a fan of both and decide from case to case which one to use. Both are under heavy development. I am often surprised about new features when checking the newest versions.
28 comments
[ 3.9 ms ] story [ 74.1 ms ] threadAnybody know of any other direct comparisons between these?
Apart from that, I actually found the comparison fair and accurate.
but mongo often downplays it's lack of durability, which makes me wary of using it or recommending it because people don't know how to use it.
Not saying that MongoDB wouldn't be faster with full commits turn on, it is just that their design decision to pick that as the default and calling their product a 'database' seemed very strange at best, and dishonest at worst.
The only benchmarks we have ever done have been for internal testing comparisons between different versions.
The "Benchmarks" page on our site clarifies this policy: http://www.mongodb.org/display/DOCS/Benchmarks
Previously there were several third party benchmarks on this page, which we recently took down to stand firmer on our benchmark policy. In the interest of full disclosure however, you may examine the page history to verify that we have always clarified these as unaffiliated third party sourced.
Well actually it depends on what type of logging. If you plan on pouring megabytes of data/sec second into it continuously you might want to find something else. I don't think CouchDB would be a good candidate for that. It might be if you plan on archiving your post-processed logged data for a long term storage but you have to be smart about its format.
Yeah 'logging' can mean a lot of things that it is hard to give a better advice without more information.
Other DBs/tools you might want to look at:
* rrdtool http://oss.oetiker.ch/rrdtool/
* MongoDB http://www.mongodb.org/
* Redis http://redis.io/
* Custom format + plain text files
RRDTool is nice, but only for data where you don't mind losing records.
For straight logging, SQL is actually a better solution than a lot of those options. If you need weird queries, something like Elastic Search or Solr on top of your SQL DB can buy you quite a bit of freedom. As can denormalizing your SQL.
One other thing to consider of course, is something like syslog.
I love CouchDB but i definitely wouldn't use it for logging.
However, a document update in CouchDB is fully ACID (including the A of Atomicity). People often confuse transactions around multiple changes with ACID for reasons that I cannot fathom. CouchDB does not have support the former (they don't scale well in clusters) but absolutely supports the latter.
I could see having a CouchDB cluster per co-location, recording a document per logging event. Then back haul through replication to a central service for analysis/dashboard use.
The only tricky part I see is to delete old logging data requires a compaction period and according to http://wiki.apache.org/couchdb/Compaction it's entirely possible for the compaction process to be slower then write data and never actually finish.
Edit: Also the back end server doing analysis may need to be a BigCouch cluster to handle the load (depending on size) https://github.com/cloudant/bigcouch
MongoDB is lightning fast at writing (provided you haven't got indexes, or your indexes fit in RAM), good at ad-hoc stuff, and a little more durable than /dev/null. Just remember to back up occasionally.
http://seancoates.com/blogs/gimme-bar-no-longer-on-couchdb
http://seancoates.com/blogs/gimme-bar-on-mongodb