Ask HN: Might as well start with SQL and migrate to NoSQL later
All of the articles I've read recently talk about the successful migration of very large websites to NoSQL solutions. The argument against are that they NoSQL is only needed for a very small percentage of websites. That coupled with the much larger SQL tools and knowledge base and uncertainty that any new website will need such scaling leads me to my question.
Why not just start with SQL and plan a migrate to a NoSQL solution later, when it's really needed?
It's certainly doable. I feel that the benefit of a NoSQL solution is, at this time, outweighed by the ease of an SQL solution for such a majority of sites that to plan from the beginning for a NoSQL solution is overkill.
Thoughts?
30 comments
[ 4.4 ms ] story [ 81.4 ms ] threada lot of these stores let you have tighter integration with your language (and its data formats), match your mental model better and sometimes have pretty revolutionary things that you just cant do easily any other way (couchapps + replication etc)
I know CouchDB is in debian stable, but it's at .8 last time I checked and >.9 breaks some things between the two. I would feel more comfortable once a solution is in the repo and basically feature stable.
http://wiki.apache.org/couchdb/Breaking_changes
First of all, you lock yourself out of a lot of useful things that other people take for granted that way. And second of all, it's really not that hard to learn.
Anecdotally I would say the opposite, that people use "nosql" stores because they do know how to use an RDBMS and they have commonly found situations when relational databases are a square peg in a round hole.
Even if it were the case, careful planning for the transition could mitigate distasteful changes.
The bottom line is to use NoSQL where it makes sense and where it solves the job better than relational databases.
If you think non-relational makes sense for your use case, try it out for smaller, non-critical parts of your system till your understanding grows. you can then decide more confidently
[0] http://news.ycombinator.com/item?id=1330552
The question is really which parts of the entire system depend on particular invariants and constraints? Where are they checked and enforced? How are the rules represented? Moving lots of rules out of declarative and into procedural code won't make things simpler.
I have experimented a lot of with all kinds of data representations, non-relational ones mostly, but I have to say, there are trade-offs that are rarely discussed in a NoSQL context. Probably because NoSQL was primarily the answer to "how do we do something incredibly simple on an incredibly huge scale?"
Many years of doing data integration have convinced me of one thing. Closely linking application code to data makes things simpler in the beginning but it leads to a completely unmaintainable mess rather more quickly than people anticipate. Data and applications have a very different life-cycle. Data lives longer and there is rarely a 1:1 relationship between application and data. That's actually where the real impedance mismatch originates, and I'm afraid it will never go away or become much simpler.
Construction workers use both nails and screws, even though they do much the same thing. It's just a question of what's appropriate for a given task.
Finally, if it's only a question of performance, I'd suggest tabling that question until it's a real issue. Scale when you need to, not before. Only once you have a "real" system with scaling stress can you actually see what needs to be optimized. Before that, it's all theory.
There are 2 things you should ask yourself. First is the scale comparable – the recipes from Facebook, YouTube, Yahoo, are not good for like 99.9% of the applications because they are not even remotely close in size and so capacity requirements. Second if this “smart thing” was truly thought out architecture choice in beginning or it was the choice within code base constrains they had, and so you might not have.
http://www.mysqlperformanceblog.com/2009/03/01/kiss-kiss-kis...
The reason is, if you're successful, the cost of migrating later on is going to be larger than you expect. You're going to be spending a lot of time doing a lot of back-end work that users never see. Once you are dealing with huge amounts of data, it takes hours and possibly days for data migrations and conversions to even run. You screw up a data import that took 4 hours... 4 hours is now killed and you have to do it over again. The more data you have, the more potential time these types of issues will absorb.
http://www.clustrix.com/
I'd go the opposite - start with NoSQL, and migrate to SQL if you need some more formalized relationships.
That said, I would suggest picking two data stores like PostgreSQL and MongoDB and write some code examples, play around with data models for your app. If you spend several hours doing this then you can make a better decision.
BTW I have tried most of the NoSQL data stores and MongoDB is probably the easiest to use from a developer's point of view. I have used bindings for Ruby and Clojure and being able to store things like Ruby or Clojure maps directly in a documents is great. That said, PostgreSQL with built in text indexing support, etc., etc. is also a great tool. I think that if you become fluent with PG and MongoDB then that is sufficient for a wide range of problems. For my development, I toss RDF data stores into the mix, but that might just complicate things for you.
Anyway, pick one of each, have fun playing with each one before making a big commitment.
Overgeneralizing, key/value datastores are useful for enormulous things that operate on Twitter/Facebook scale and handle loads measured in thousands of operations per second. Looking at a few existing services that need that sort of throughput, you'll quickly notice that they're all backed by tens of millions of VC and have big teams they can devote to dealing with the overhead needed to build Big Scalable Things.
On the other hand, if you look at the dozens of "little services that charge people money to use them" that the typical bootstrapped startup founder should be striving to build, you'll notice that they all would work just fine on a relational database. So given that there are tools to get you shipped 10X as fast on an oldskool RDMS, it's surprising to see that ANY of these lean little startups would choose to go with NoSQL right out of the gate.
Due to the vast number of libraries for any language, and the vast number of helper/managers like ActiveRecord or DataMapper, I'm surprised nobody is advocating them more as they do support the "lean, fast, daily updates" while still small. When you need a higher performing system, it's available, but take advantage of all the previous library research. You're doing a disservice if you spend too much time to ship on exotic system, when you don't need it for another 1-3 years.
What real benefit is there starting out with NoSQL when most websites really don't need to scale that big.
Redis, for example, is much more about making single boxes perform like crazy, rather than massive scaling (although they are working on a clustering for version 2.0).
Cassandra is mostly about the scaling, but is also very fast (though not as fast as redis), and if you set it up on three nodes you've got high availability. Since I plan to keep my day job, I can't drop everything to deal with server issues. I want something that just keeps on trucking no matter what.
This might mean using a NoSQL database. MongoDB seems easy to try out, it could be something you'd use for some component.
But most projects fail before they ever reach some minimally useful state, not because they don't scale. Plan ahead for scaling where appropriate, but the main focus should be getting it out there in a useful state.
The real advantage will be when I get to the point of seeing real traffic, I can scale horizontally on inexpensive machines. I'll be able to maxmize all parts of physical machines, when I make to the move to maintaining my own infrastructure.
Honestly, I'm not sure why I'd want to use SQL at all, especially for a project in it's infancy. With many NoSQL implementations you can adjust your schema as you go, instead of locking yourself early into a schema that may not fit a few years down the road, and it becomes a very difficult project to make changes to it.
The caveat being that everything I'm doing works with with a key/value table. My data set and how I access it wouldn't take advantage of a lot of what SQL provides. So, if your application does benefit from SQL, then by all means use it.
For a bootstrapped startup nosql makes total sense b/c you get much more performance for your dollar.
Sure you can go the vanilla PHP/MySql or Django/Postgres route if you want to build your prototype but you will be in a bad position if your system can't scale with demand.
I would build the prototype first in your favorite framework/setup.. If it has market potential after some initial private beta testing I'd definitely go the nosql route.