Kudos to the team; some great releases this summer! (10x insert performance, regex queries, and now date/time and better disk space--all in ~90 days!)
I had never looked in to how rdb used disk space and off the cuff attributed its high usage to something similar to Mongo and allocating blocks ahead of usage.
This was something that went in to the negative column for me when I evaluated using rdb vs Postgres for a JSON dump. 12k somewhat light (but still larger than 250b) records ate up over 70MB of disk space in just over a couple days and I've not looked back. Time to give it another spin!
There is no technical reason why Rethink needs to have high disk usage. We got much better with this release, and will continue to get even better in the long term. (Squeezing more optimizations does get a bit harder, though)
This is good to hear and will make me try it too. High disk space utilization was something I experienced with mongodb back in the day and that caused me to lose interest (not sure if they're better about it these days). There's probably a trade-off between speed and having blocks allocated but it starts becoming a problem on smaller vps environments.
Mongo got a lot better about reusing disk space in v2.2, if you are deleting from a collection. Still no compression or document field name optimizations though.
I did a small project so maybe someone that did something larger can weigh in, but I worked with RethinkDB back in the 1.4 era, so a bit has changed, but the biggest difference IMO is that you have to be prepared to fall back to the RQL syntax to do queries more complex than simple get etc. vs. in Mongo everything was in JSON, and when I was using Mogno you couldn't join tables at all, don't know if that's changed and here you can. Otherwise it is pretty similar, since I didn't do any clustering.
Can you please include the entry on connection management model. PSQL is known for taking process-per-connection + shared memory approach, knowing the canonical and "mothership approved" approach to handle Rethink connections would be awesome.
I keep thinking that I should try RethinkDB, but most of my projects involve geodata / geospatial indexing. For this use-case PostGIS and MongoDB have benefits which I can't find elsewhere.
We have a github issue open for it FYI (can't find it now because github issue search seems to be down), but unfortunately it won't get done before 2.0.
Do you mean a Wikipedia article? If you mean a wiki for users, we use the Github wiki (https://github.com/rethinkdb/rethinkdb/wiki) to track things like community contributions.
Great Job and this is cool to see such progress. Now the long process of implementing all the new verbs in the drivers begins. I am really looking forwards to when RQL is marked as stable.
Tables are a little bit heavy now. I don't know the exact overhead (we should measure), but a table is certainly heavier than a single document. I'm not sure if a table/client is tenable -- if end up doing some tests could you please report the results?
So no new verbs going into the Query Language ? I have been waiting to do much more work around my Java driver until the pace of change slows down a bit.
What.. version 1.8 and now there is Date support? Shouldn't this kind of stuff be aournd at version 0.01 when we are talking about data stores? You know, the only non-easily swappable piece of our tech stack?
It's really comments like this that make me scared to death of posting a project to HN these days (so I don't). Seems to be so common now.
It's getting quite common for me to just skip reading the user comments after reading the article. This was never the case before; the comments were -almost always- better than the article.
As a sidenote, I cannot even recall when I actually read the user comments on Slashdot. On the occasion where something pops up in the RSS feed; read TFA and move on to happier things that actually make me feel good. :)
I guess I'm not saying anything new. I really wish I had a solution other than asking -you- to please imagine sitting at the receiving end: You get excited to see your project on HN, you click through to comments, and the first thing that was said was ... that.
PS. You can call me a chicken for being "afraid" of these things.
Sometimes people get angry and irritated at stuff. That's ok. The tone is uncalled for, but irritated/angry comments often have legitimate gripes. Rethink should have had dates earlier, we just couldn't get it done.
I agree that people should try and be more kind with their tone, but I wanted to try and encourage you not to be afraid of criticism. People have good memory for good work and no memory for bad work. So if you post an atrocious project here, the absolute worst thing that might happen is that some people will write some negative comments and forget about you ten minutes later. It seems like a worst-case scenario most people can live with :)
It should have been, but date support is way, way trickier than it appears. Check this out if you want to learn more -- https://gist.github.com/coffeemug/6168031. It's bad that dates didn't get in earlier, but it would have been worse if we rushed a bad implementation. Sometimes one needs to pick the least of two evils.
I've been watching the great progress since 1.4, but I continue to be disappointed by the overall performance, makes me wonder if I'm not groking something. Here's a gist of a simple ruby script to insert 1000 trivial docs into rethink:
You have something wrong there. I just ranthe test suite I ship with my java driver against and I am able to do 2048 documents in about a second. Most of that time is spend in serializing the data to go over the wire and each insert is being done individually and in raw protobuf form. If you send in JSON and you queued up your inserts I imagine you could do even better.
Performance tuning never stops. We have one more release coming up that will squash a lion's share of low-hanging performance fruit. After that, we'll be tuning continuously, but it will be slow, incremental improvement.
Keep up the great work. It's a very interesting project, one I'll continue to keep an eye on.
One thing I wish would get added are more aggregators. This would add a lot of valuable use cases, in my opinion. At least one of my mini projects recently would have been greatly simplified had I had the ability to compute max's and min's in the same way as avg's.
Look, I don't have enough experience to be sure on this kind of thing, but it seems like RethinkDB is being developed at a blistering pace. Docker is another one like that. Both impress me, both on their vision and rate of improvement. It makes me happy to know that there are successful projects like this out there.
Lack of native Date support was holding me back previously. Now i don't see any reason for not using rethinkdb.
My next project will be using Rethinkdb! It will be nice if you can fix "groupby field names" issue https://github.com/rethinkdb/rethinkdb/issues/386
in your 1.9 release
50 comments
[ 2.6 ms ] story [ 116 ms ] threadI had never looked in to how rdb used disk space and off the cuff attributed its high usage to something similar to Mongo and allocating blocks ahead of usage.
This was something that went in to the negative column for me when I evaluated using rdb vs Postgres for a JSON dump. 12k somewhat light (but still larger than 250b) records ate up over 70MB of disk space in just over a couple days and I've not looked back. Time to give it another spin!
I feel more confident that the m will be easy to scale if/when my app needs it.
My code that interacts with the API feels cleaner.
Can you please include the entry on connection management model. PSQL is known for taking process-per-connection + shared memory approach, knowing the canonical and "mothership approved" approach to handle Rethink connections would be awesome.
Thanks!
https://github.com/rethinkdb/rethinkdb/issues/1158 ?
It's getting quite common for me to just skip reading the user comments after reading the article. This was never the case before; the comments were -almost always- better than the article.
As a sidenote, I cannot even recall when I actually read the user comments on Slashdot. On the occasion where something pops up in the RSS feed; read TFA and move on to happier things that actually make me feel good. :)
I guess I'm not saying anything new. I really wish I had a solution other than asking -you- to please imagine sitting at the receiving end: You get excited to see your project on HN, you click through to comments, and the first thing that was said was ... that.
PS. You can call me a chicken for being "afraid" of these things.
I agree that people should try and be more kind with their tone, but I wanted to try and encourage you not to be afraid of criticism. People have good memory for good work and no memory for bad work. So if you post an atrocious project here, the absolute worst thing that might happen is that some people will write some negative comments and forget about you ten minutes later. It seems like a worst-case scenario most people can live with :)
https://gist.github.com/wiremine/6245910
1,000 inserts are taking around 20 seconds. Hardware is a 1.8 Core i5 Mac Air with 4GB Ram and a 128 SSD.
Is this sort of performance expected at this point?
13.618112323 seconds vs 0.591895712 seconds (on my laptop -- X1 Carbon)
Using soft durability make things faster too, but that's cheating.
One thing I wish would get added are more aggregators. This would add a lot of valuable use cases, in my opinion. At least one of my mini projects recently would have been greatly simplified had I had the ability to compute max's and min's in the same way as avg's.
Thanks for the feedback : )