This post makes the assumption that MongoDB isn't doing well based on Hacker News. MongoDB the company, and product, are doing extremely well (outside of anecdotal Hacker News posts), based on objective metrics like downloads, skills, jobs, and company revenue (and the company is nearing 900 employees on linkedin).
To put it another way, RethinkDB did extremely well on Hacker News. Twitter didn't, if you remember all the negative posts (and still went public). There is little relation between success on Hacker News and company success.
Can't speak to current popularity, but I think the overall critique is spot on. It became really popular riding a combination of up-front friendliness and the No-SQL hype bubble. The next morning, a lot of folks started feeling the hangover, whether from actual problems with Mongo or a realization that they did, in fact, need a relational DB.
To be clear, I've only ever run it as support for something else that depended on it. It just isn't well-suited to my projects and it wasn't interesting enough to invest much more than noodling a bit with it. I'm sure it is a fine product that turned in to a victim of its own hype.
It does serve as a decent example for the next generation of developers about how flavor-of-the-month trends in tech make people do stupid things, like decide they don't actually need ACID for handling money.
MongoDB is like the PHP of databases. I started programming PHP and currently I bash it every time I can, because it is a terrible language. But it is easy to start and do cool stuff, just like mongodb. Initial traction is more important than initial architecture, I guess.
Why is PHP a terrible language? It powers a vast part of the internet including sites huge, tiny, and everything in between.
It's in vogue to hate on PHP especially here on HN. Language hate based on some sort of popularity contest is silly and language snobbery wont be tolerated at any company worth its salt (read: NOT most startups)
This lack of insight is why PHP developers have lower average salaries than non-PHP developers.
If you think PHP is looked down on by those who care about language features because of a popularity context, rather than because of asinine language decisions and limited support for modern programming paradigms, I think it's you who is missing it. Since any program can technically be written in any (Turing Complete) language, the fact that some big sites were made to work with PHP also sort of misses the point. When most of the big "success stories" adopted PHP, options for web-friendly open source languages were much thinner. Now if you look at the new development projects they highlight, you don't see PHP come up that much. Facebook, for example, is prominently using Haskell. I don't know anyone who both programs in Haskell and is willing to touch PHP.
You're not wrong, but those articles are 5+ years old and PHP has changed A LOT in recent years — especially since the release of v7.
Still, that's not going to do much to change public perception, and I guess that's somewhat fair. PHP has earned it's reputation, even if many (but not all) criticism are now largely dated.
I'm just happy I switched to working on other languages, aside from the occasional Laravel-based side projects.
Thanks, that is absolutely worth pointing out. But there is only so much you can do to fix languages when they are designed with problems initially and you want to keep old programs working. I have the same concerns about ES6 BTW, although JavaScript foundations are arguably less flawed overall (excepting a few areas that are just bat shit crazy).
Is public perception the reason you are happy to be mostly using something else now, though?
It's certainly nice when talking in developer circles, but no, public perception is not the primary reason — it's more of a bonus :)
I really enjoy some of the more flexible syntax and design choices present in some other languages. It also help that various job requirements over the year have required that I build a broader skillset. Finally, the growth I've experienced by stepping outside my comfort zone the past few years is probably the best benefit, which is a good experience regardless of which language(s) you start with.
In the end, however, as we both know, most decisions in software development are an exercise in managing tradeoffs. I guess the core of my argument is that, in the past few years, PHP has done a great job at improving its balance of tradeoffs.
> PHP has changed A LOT in recent years — especially since the release of v7.
I was curious and wanted to see if that is the case. So I checked for the most basic thing. The arrow notation for lambdas. Surely they must've added that by now. Surely they must've added that before Java did. Nope, still RFC. Now I wonder if it's idiomatic to pass those "long form" lambdas around when writing in functional style or is it something nobody does. In any event this little discovery makes me think parent's sentiment seems to be still justified.
It's not "in vogue". There's no trend here, the PHP hate has been very consistent over time.
Personally, having had to write a couple of Mediawiki plugins and an extension for a rubbish CMS called TYPO3, my experience so far is that the PHP hate is very much justified.
Seriously, the biggest problem with php is that fundamentally it's a templating language. One thing I've seen with templating libraries in general, is that they are meant to take a template and a data structure and display output based on these 2, but what ends up happening is they end up converging on turing completeness. It starts with the need for some kind of looping structure in the template. And then someone needs some random logic that is easier to do in the template than before the model, and so another feature gets added to the templating engine. And so it goes, things get kind of bolted on, without a grand vision, and you end up with an API that is inconsistent, where the order of inputs is never the same (is the first parameter haystack or needle??).
Also, the basic shape of the code that tends to get written in PHP is heavily influenced by it's being a templating language. PHP applications tend to end up having a lot of code in the template, that should be done before rendering the model. Rather than a sane 2 pass system, where all the data is gathered and the logic handled to produce a simple data structure that is passed to a very simple template, you end up with crazy monstrosities where there's opening and closing <?php> tags everywhere. And you render part of the page, then grab some more data, then render some more page, then maybe do a curl to a 3rd party vendor, then render some more of the page, and it's actually very hard to keep switching mental state to match the context of the code you're looking at.
Anyhoooo, you're getting downvoted because there's lots of literature on why PHP is a terrible language. I'm only replying because it's sort of cathartic, since I maintain a wordpress codebase, and any time I have to interact with core, instead of being able to sanely intercept the underlying data, modify it then pass it along in one place, I have to constantly add lots of random hooks in lots of random places to catch the data just before it's being rendered. It's a total mess, and far more complicated than it needs to be (though there is some debate whether this style made it more approachable from a novice plugin development perspective).
You described exactly what I'm experiencing. I've had this awkward feeling that was hard to describe in my current work and you were able to express my exact thoughts in words. Thanks for sharing
Way back when I started reading HN, one thing that really struck me was seeing PHP discussed in a practical fashion rather than merely bashing it whenever it's mentioned as was the case in Proggit those days (often, the bashers obviously having very little experience building software).
By that time I was doing most of my coding in Ruby and Python, but this businesslike approach to PHP made me realize this is a place where people really get their hands dirty and build things, rather than care about fads.
Things have really changed, the tone & level of discussion here are now very different. Seems like PHP is kind of a canary in the coal mine in that regard.
What are you talking about? PHP has been rightfully bashed ever since it was created. If anything the bashing has substantially gone down in the past years as sane array notation has been added and a lot of weirdness has gone away.
> It powers a vast part of the internet including sites huge, tiny, and everything in between.
It's more like those vast parts of the internet are working despite PHP. Why do you think Windows is powering vast numbers of desktops? It's not for its technical excellency. ;)
These days MongoDB more or less (finally) works as advertised as well. It actually has worthwhile backend options (plural!) and the new (v1 vs v0) replication protocol is actually correct. Eight or so years of development has led to the most egregious of the bugs and corner cases being rounded off.
So, it started as a shitshow on the back-end and people who had to maintain systems based on it hated it, but it got popular because it was easy to get simple systems up and running with. Then, over time, it slowly became less awful as it was substantially rewritten and large portions of it redesigned. I'd say PHP is a very, very good comparison.
Well, Facebook and Wikipedia are using HHVM to execute their PHP code, so at least those two are not really satisfied with PHP. I bet they'd like to rewrite their code in a faster, more memory-efficient, language but are hindered by the complexity of their code base.
A lot broken-ness is still left for the sake of compatibility (especially since HHVM's open-source), but Collections and type hinting make a huge difference.
That Slack post is terrible. I can't believe they would site concurrency as a strength of PHP...."Asynchronously curl’ing to localhost (or even another web server) provides a shared-nothing, copy-in/copy-out way of exploiting parallelism."
I almost have to wonder if this is satire
Wikipedia launched in 2001. Facebook launched in 2004.
The choice was narrower then. Ruby and Python, while already active, did not have nice web and DB frameworks (both RoR and Django started in 2005). Java was also much less mature, and even more wordy.
PHP, on the other had, was easy to get from any shared hoster.
I understand the importance of well thought out back-end architecture, and that it isn't my realm. I just have to say that I really enjoy building things in the MEAN stack, and had fun learning MongoDB.
For a long time all I had ever done was relational, but then my company started using Azure table storage for a lot of projects and I became familiar with other models. Then I discovered Mongo while catching up on Angular, and enjoyed using it with the mostly small apps and prototypes I build. I don't know it well enough to try and support it's use on a large scale. After seeing how poorly designed the data is for so many companies out there, it's not a surprise to hear poor choices were made (and probably some poor execution).
I don't know. I was just in a meeting where a team announced that they would need a ton of money and time to move off of MongoDB. The VP reminded them of when they had chosen to move to MongoDB years ago and asked them what happened to all of their promises about how great MongoDB would be. They responded that the promise of MongoDB and the reality are very different and MongoDB has turned out to be a disaster.
10gen is not a company dedicated to databases, at least not if the initial code quality of MongoDB is a signal (and it should be, since if you know a thing or two about storage you wouldn't take the compromises it initially did)
Unfortunately, many fresh graduates with no experience in storage decided to evaluate the technology based on its very visible merits, without considering that scalability is not really a top priority even in companies that have to "scale", and that reliability trumps every other metric.
Mongo has failed spectacularly in being a reliable data store, and its scalability is mediocre, given that it's a lot more about query planning, analytics and optimization than having out-of-the-box sharding.
This is one of those areas having some seniority does wonders. Storage concepts are not easy to grasp, and evaluating a database's reliability takes much longer than most product cycles.
The hope is that new technologies would provide some advantage, like prototyping speed. That was certainly true of Ruby on Rails, for example, back when it was the new kid on the block and the "tried and true" frameworks were stuff like Struts or Spring. Yes, that's not a database technology example.
Have you actually studied the JSONB capabilities of PostgreSQL and related features such as partial indexes? These things can be used at scale and there is at least one company that has millions of partial indexes in production.
And you might want to look at how SalesForce has implemented a highly scalable architecture on top of Oracle RDBMS servers. That includes a document store as well as a dynamic denormalized schema layer that works real well for reporting.
Heck, I've implemented a persistent memcache server on top of SQLITE and it scaled pretty well because modern servers and network infrastructure just happen to do most of the work needed.
It is not easy to learn PostgreSQL well enough to evaluate it, but more and more people are doing so and finding that it is the ideal open source persistence solution for the vast majority of business uses.
Honestly you can say any of these things about MongoDB. Their marketing team sure does. Did you know MongoDB is web scale?
I'm not insulting Postgres (actually a big fan), but Postgres added JSON support after. By that metric, document store in Postgres is less tried-and-true than MongoDB. Does that mean it's the wrong choice? No. Age of product is not always a good qualifier.
I'll take "less mature" over "data loss" every day and twice on Sunday.
Whatever else you might say about Postgres, that community cares about the integrity of its users data, over pretty much every other consideration. I don't think the same can be said of Mongo, or any other "NoSQL" anything.
You can reimplement 95% of the "document model" with a relational database just fine, and you'll actually come out ahead given all its advantages.
Unstructured documents are very rare. Querying arbitrarily against them even more so. For the fields that do need querying in a document, put them apart and add a few indexes and you're good. That's been my experience and I've been around a while.
Not only that but you can use PostgreSQL as a NoSQL and there are benchmarks showing that it outperforms MongoDB.
And if you do not want to commit to either SQL or NoSQL, the PostgreSQL has you covered. You can put a JSONB column in any of your tables to handle the part of your data that is schemaless, and you can still index the data in that JSONB column.
There are a very few applications that need distributed scalability beyond what Oracle or PostgreSQL can provide, and those ones need NoSQL. They also need highly skilled competent developers who are capable of reimplementing some needed functionality of RDBMSes at scale.
These PostgreSQL features are relatively recent. The jsonb column type dates back to 9.4, which was released in Dec 2014, less than three years ago. MongoDB had been around for well over five years by that point.
There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
Yep, you can't do stuff like update JSON in place until PG 9.5 (without resorting to a crazy regex, that is).
You also do have to think at least a little about your JSON structure given you can't really use an ORM to query JSONB, and the syntax for JSONB is a bit awkward at times.
> These PostgreSQL features are relatively recent. The jsonb column type dates back to 9.4, which was released in Dec 2014, less than three years ago. MongoDB had been around for well over five years by that point.
It's much much harder to design a reliable engine than a new datatype.
> There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
Yes they are, but chances are you don't have that need. You're not Google or Facebook.
You also picked up a really bad database to talk about scalability[1]
SQL is a lot for newcomers to grasp on top of learning the rest of their stack. The appeal of "no schema" is most likely what gets them as it seems easier than SQL. Simple things at first seem daunting like: constraints, joins, and indexes. What they fail to see is that doing these database level things in the application layer becomes the final level hell and you get to live it.
People also undervalue their data and it's integrity.
I personally do not like mongo and see zero reason for it except perhaps in some very niche use cases and even then, I'd still choose another document store over it.
Maybe I'm too old now... but SQL is relational algebra translated into some sort of code/language. I did a lot of SQL & relational algebra for my undergrad databases class [which was about 20 years ago now]. Has SQL/DBs been phased out of CS education? Just genuinely curious...
I have an intern who's bout to graduate from a state university and has taken 0 classes on the implementation of compilers or databases. Java, Java. Java.
I'd say don't worry: I almost never use anything I learned in school.
The Java was to shallow as was lots of other stuff.
School is just there to convince your first few employers so you can het a chance to see how it is actually done.
Oh, and to give you the credentials to stop annoying sales guys that are about to convince your boss that "you need Active Directory because your network is slow". (Guys company had botched a certain server install and tried to solve it by selling even mire stuff. )
I graduated from an above average university in the UK in the last 5 years and we had multiple modules covering concepts of relational databases, data warehousing, hands-on MySQL and Oracle including full-stack application projects. There was a lot of Java and little more than a Hello World in C
I think a web development class I took in college a few years ago spent maybe two or three class periods on SQL. Felt like enough to get by with, and learn more as you go.
I finished my BS in CS 7 years ago. I took an elective class on relational databases. We spent about the first 25% of the class on relational algebra & SQL, and then used SQL fairly consistently for the remainder of the class.
I graduated 7-8 years ago and we did a lot of SQL so I'd surprised if that's not the case today. I think one or two SQL/relational database classes are almost a norm.
A huge amount of people working as developers have no formal CS education whatsoever. In my company I even hear it touted as a perverse point of pride sometimes. (Kind of like, Oh you have a degree? I studied biology and I can still do the same job.)
I have a CS education, but have no real experience. So now, as a biomedical researcher, trying to rapidly prototype for medical devices and developing software is extremely daunting. I don't think these prideful people understand just how far a formal background can go to speed up the process when things get more complex.
Nope, my college had at least 2 classes to cover databases. First one was to learn basics (although it had fun project to implement a DB engine and handle simple SQL), and second one was more about using it in real applications.
The classes were not mandatory so you could technically graduate with CS without knowing anything about databases.
Have you ever worked on a significant project using MongoDB?
I did. I wouldn't use it again. But reliability just wasn't an issue. There was a time when there were insane defaults that made this an issue, but that's a long time ago.
MemSQL as in the distributed relational data warehouse product? If so, it's highly recommended, probably the best data warehouse solution you can run if you're not going with something managed (like redshift or bigquery).
Feel free to reach out to me on our public Slack channel to clear up anything that is fuzzy! I'm an engineer at MemSQL and want to know (1) how I can help and (2) how we can provide non-fuzzy information to the rest of the community.
I'm not a back-end dev strictly. I played with mongo on some side projects back when it was all hyped up. One thing that appealed to me was the on-boarding process for a non "database guy". I got lots of small side projects up and running very quickly and the concepts made sense. Similar to jQuery being easy to understand and popular to newcomers but you later learn it isn't always the best tool for the job.
It isn't an age thing. There are mid-life engineers where I work who are constantly trying to force the latest fad in tech into our engineering organization.
This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex.
No wonder the whole javascript stack is a complete and utter mess!
> NoSQL originally meany to not use SQL as the query language
I'm pretty sure that's wrong and it always meant not to use the relational model; it was only “NoSQL” because it's short, easily memorable, and there's a percieved (though not exactly true) equivalence between SQL-based and relational databases.
ACID and relational are orthogonal. You can have ACID properties without a relational model. Much of the early literature on the subject merely discusses databases in general rather than any particular type. Here's an example from Jim Grey:
I still think MongoDB is a good database. Conceptually, It's not as good as RethinkDB but it's not too far off. It's good to not have to deal with SQL and horrible ORMs anymore.
TBH, I'd rather write SQL directly rather than use an ORM though. I never want to use an ORM ever again. At least that's one problem that MongoDB solved.
Mongoose is a lot simpler and less brittle than other ORMs in SQL land. That said, for most use cases, you don't need to use Mongoose.
When it comes to Mongo, I think it's nice to have the DB schema documented somewhere but you don't really need to have it enforced with Mongoose... It introduces more problems than solutions.
With Mongoose, you have to run migrations on your data more often which slows you down. Though it might make sense for bigger companies.
1. Architecturally is was very familiar to traditional relational systems like MySQL. Meaning classical master/slave, sharding and absolute consistency via locks.
2. Mongo's rise coincided with the rise of NodeJS and continued on the success of Ruby on Rails.
3. Mongo has done a phenomenally fantastic job attracting developers with exceptional documentation and getting started materials.
These all combined into a perfect storm attracting legions of developers. Note that these things have absolutely nothing to do with the technical capabilities of Mongo as an actual database.
Not so different in some ways from the rise of MySQL itself.
In point #2 substitute PHP for Node and RoR.
In point #3 just change the name from Mongo to MySQL. MySQL AB was brilliant at getting MySQL into Linux distros. When you came to need a DBMS to go along with PHP chances were you could just type 'mysql' and you would connect to a live DBMS instance on the host.
The 10gen team also imitated the MySQL playbook explicitly with built-in replication and an early lack of focus on ACID properties.
Back in 2012 I worked at a startup and we had a few backend data stores (Vertica, Postgres, Redis, Big Table, MySQL, Cassandra, ElasticSearch, Hadoop).
When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We wouldn't have to write SQL anymore (which was actually funny to watch one of our engineers try to unlearn table denormalization).
At the end of the day, it was hype. I had to stay up during multiple 2012 debates for 5 hours and flip the Mongo databases because they kept crashing. The shard key they set up for us was slightly out of balance so the data distribution was 30-30-40 and every so often, that server processing 40% of our data would jump to 50% and then knock that server offline leaving the other 2 to split 50% of the work... knocking them offline. There were also tons of replication problems all traced to write locking. At the end of the day 10gen solved one problem, but our company traded that solution for other problems.
That experience taught me that you really need to understand what you're trying to solve before picking a database. Mongo is great for some things and terrible for others. Knowing what I know now, I would have probably chosen Kafka.
You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft.
I'm sure it's better today. I'm talking about my experience back then. It may be a challenger, but it still has tradeoffs that system architects need to factor in when making database decisions.
I think the worst part about MongoDB in that timeframe was the poor CPU utilization since it was single threaded.
In order to leverage the multiple CPUs in a box you'd need to run multiple instances of mongo on the same machine and we designed a whole system around sandboxing instances.
But then you'd run into memory allocation issues and how badly it would handle indexing and handle running out of memory, which mostly involved crashing.
We ended up having to run about 100 bare-metal servers to support what would probably have been a level of queries about 10 PostgreSQL boxes could handle easily.
Any criticism of MongoDB that does not acknowledge WiredTiger and the plethora of positive changes that made MongoDB much, much better in the 3.0, 3.2 and 3.4 releases is criticism that is not fully thought out.
It's like complaining getting graphic cards working because of drivers is difficult on Ubuntu, based on experience with linux in 2005.
In the end it doesn't really matter. I feel the article is an educational one, but as it's the chance for us to take our sides of the fence in praising and criticising, we'll still do it anyway.
I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days.
A lot of software fails, companies close, while some succeed. Some software that has been hyped up ended up not succeeding, but it doesn't matter. One learns from others' mistakes and keeps growing.
You realize that this sounds like more of the same hype to anyone who was burned by the overpromising in the 1.x and 2.x days? Whether your statement about 3.x is true or not, the MongoDB people dug their own hole that they need to try extra hard to get out of, and so far it's not enough.
Not to sound flippant but the reason why so many startups choose mongodb is because the problems they are working on are better suited to be addressed with a document-oriented database than relational database.
And at the time, most of the mindshare is/was with mongodb for document-oriented technology.
I've assessed a fair number of companies who use Mongo in some way or another and the I always ask the rationale for the decision. Surprise surprise, it's usually "we chose it because it was easy to start with".
My favorite recent one was a company that had to persist their Mongo database into AzureSQL (nightly job) so they could then push it to Good Data for analytics. They could have very easily used SQL Server and pushed directly from there to Good Data. And yes the company admitted that was the case.
In case of Edmunds, the rationale was that their contractors previously used Mongo and were more familiar with, and upcoming 2.6 version will have some "new cool features".
Also the table is misleading, PostgreSQL replaced Oracle, Mongo replaced Oracle Coherence[1].
Looking at the table from 10th Gen, I believe it is misleading, at least in relation to Edmunds. I worked there, and Oracle (which was the source of truth) was replaced by PostgreSQL.
Mongo did replace Oracle product but it was Oracle's Coherence[1]. There wasn't a concern of Mongo loosing data, because the data could be reloaded again from Postgres if it got lost.
I also heard that they were considering replacing Mongo with Postgres as well, but I do not know where this went.
I think 10th Gen is just good at generating hype. The migration was not because Mongo was better but because Oracle was expensive and not that much better from Open Source alternatives.
I think the short answer is the same today as it was back in those days. A person once put it to me simply: "programmers are afraid of databases."
It was a perfect mixture of this unfounded fear with an advertised simple solution. Add the idea that there is no need for a DBA, encourage developers to see Mongo as a magic data storage and nothing more, and you have the perfect product for those who don't feel the need to look deeper.
Startups seem more optimized to fast iteration and quick development. It seems to me that they chose Mongo because they prioritized iteration and ease-of-development over the hard-nosed style needed to use a relational database well.
Maybe it's an age thing (or the product of "bootcamps"), but I don't think that's true. I'm 40, and when I was learning, you connected to and wrote SQL against the database. These days I still spend a significant amount of time writing SQL, both for ad hoc queries and for complex queries / performance optimization.
> Add the idea that there is no need for a DBA
With many cloud database offerings this is essentially true. As for things like indexes, etc... developers targeting Mongo need these as well.
> encourage developers to see Mongo as a magic data storage and nothing more
In modern frameworks, this tends to be true no matter what the data store is.
Interesting article, and I look forward to the other 2 parts. This was well-researched and balanced.
MongoDB feels like a different product to what it was when the hype was high. The company, leadership and engineering quality certainly have changed.
Someone else commented about the HN bias, which we can't deny exists.
I've been running an 1-member replica for as long as I could figure out how replicas work. So I've had my Mongo database running for the last 5 years. It's met the sweet spot that I was looking for when I knew less than what I do today.
Even if one were to rubbish the product, it still has value as the company seems to be doing well (even with the negative sentiment and press).
Mongo sometimes feels like how "corporates" (at least in the recent past) feel about open-source. Dealing with our SAS account at work, the sentiment used to be that open-source was a waste of our time. Open-source is brittle and slow, rather use Oracle, etc. SAP HANA will change your life, in-memory is hard, our "appliances" run on unsecular hardware ...
Those sentiments have changed over time, so we shouldn't eternally write things off if they're healthily maintained.
I look forward to mixing SQL and NoSQL well into my future. DocumentDB wouldn't be what it is today if there was no Mongo . The SQL vs NoSQL flame wars get us nowhere to be frank.
The company got bankrupted and they set up an org to fund its development. (Sagemath moved to Postgresql from RethinkDB after the company got sold. The guy behind sage said it was better in his blog about the switch)
Incidentally, http://www.lmfdb.org uses MongoDB (since maybe 2011) pretty heavily, as a result of me convincing them to do so after I read too much about MongoDB back then. It’s been painful for them, but it does still work well...
RethinkDB did a lot of things right from an engineering perspective, but they were too late to market to compete with MonogDB; they burned through their cash, then went out of business, and now the devs work at various other companies (e.g., Stripe). RethinkDB is not a viable choice today.
I personally put enormous effort into trying to use RethinkDB (and Cassandra, and also MongoDB); now I very happily use PostgreSQL. just like I did back in the 1999, and I’m very happy with PostgreSQL.
There were so many blog posts about how MongoDB was the future and SQL/relational dbs were the past.
I wish someone would collect all the hyped up tech blog posts so that we always have a reference to be able to say, "see what hype does?" Sadly, I think it would only increase the number of devs saying, "Not this time, it's different."
Hype and lack of critical thinking abolutely had their role, but mostly because it's so much easier than a relational db.
Relational db's are hard. You have schemas and query planners and the truth is, you have to take care of every single query. Every query counts.
The same query can take 30 seconds, or 30 milliseconds to run, depending on it using an index or not. You think it's ok for it to take 2 or 3 seconds, until it hits production and halts the server.
Mongo doesn't have any of that. Just load and dump json back and forth, which is perfect for fast prototyping.
It has most of that. It's not that RDBMS are hard and schemaless convolutions are easier.
Mongo has query planners, the same thinking you do in SQL applies to Mongo, the edge cases are different. For example, you can't have composite indices on multiple arrays.
The same principles we consider when tuning our RDB apply to Mongo, normalisation comes at different forms/levels. The same applies to denormalisation. Just because there are/were "no joins" didn't mean that everything goes into a single collection.
I run some Mongo deployments, and I take as much care in understanding performance, as I do with our SQL deployments. Databases are hard to those who are unwilling to open up the manual and read. It's not only SQL.
MongoDB is an interesting database for sure. It seems that they made a deliberate product decision to optimize developer experience and ease of use over things like consistency and database joins. Frankly, I think it's working pretty well for their business.
I don't think there's anything that should immediately preclude the use of MongoDB in a production environment. I personally wouldn't use older versions of it as a single source of truth, but there are definitely some great use cases for it, such as allowing developers who only know JavaScript to write non-critical backend services.
People seem to forget that database consistency is not binary, but residing on a spectrum. Like any other trade-offs, you should carefully measure the costs and benefits for your particular situation before making a decision.
And in turn put the schema (actually multiple versions of it) in your application making it more complex.
Since Relational Algebra was invented in 1970 we were unable to find a better representation of a relational data, there were many attempts but so far they failed (except for very specialized uses). So I'm inclined to believe it's here to stay.
JSON is basically just step backwards back to the XML Database[1] and Object Oriented Database[2]. It starts falling apart as soon as your data becomes more complex (you have relations, or need to query it by something else than primary key). Having said that, I'm glad that relational databases added support to it, it perhaps might be useful in early stages when things are in flux.
I have been paid by two companies to remove Mongo. One that was hosted at the Mongo HQ. (The mongo HQ guys are really nice). At small startups, things are on fire and people need answers quick. "How many users tried this, how many times was feature x used," on and on. If it is in mysql, well i just setup a vpn and a read mysql access, done -- they can query away.
If it is all in mongo? Welp. How is your python? "See there is this key, and you need to sum this -- OH and remember that this does.. Hmm, "
I use Mongo & its been working great. Our data is not relational (imagine you were tasked with building "Powerpoint" on the web).
Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over.
With something like mysql I have to learn about & choose between row based replication or statement based replication. I have to choose between binary logging or global transaction ID logging. With Mongo I just call rs.init() and rs.add() and it "just works".
Perhaps I'm just not hitting the level of traffic where the problems begin to surface, or maybe postgres would be better... but its been great so far. No complaints. The only issue was de-normalized data getting out of sync due to bad coding in our app, which i do not blame Mongo for (and was simple to fix).
Of course I don't know your product, but when I imagine Powerpoint-as-a service, do users not relate to decks which relate to slides which relate to the content on those slides? Content itself of course would probably be a pretty flexible tree structure.
An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching for indices and foreign constraints across what definitely feel like relations to me.
And for the non-relational content blobs, Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images", "who are the top 10 users in terms of edits to content tree items, and what subscription level do they have" pretty neat and fast.
If you're in early stages such that you don't want to hire or acquire DBA skills, you're probably well served by offloading replication/backup/verification (which are all very possible with RDBMS) to e.g. AWS's RDS product.
HA is important, is covered by replication+failover. Periodic backups are important, are covered by a export+rotate schedule. Point-in-time restores are important, are covered by storing replication logs. All are covered by a managed RDBMS, so you need to know what they are and how they work, but not implement or manage them. Most of the NoSQL deployments I've seen just rely on the out-of-the-box replication, and get HA but that's it. The engineers I've seen that don't want to understand the simple high-level ramifications-to-developers of RDBMS replication don't want to understand ramifications-to-developers about their NoSQL replication either, and they take a risk of encountering those ramifications needlessly unforeseen.
As the data grows, yeah, maybe you'll eventually hit technical or economical limits that mean you'll want to manage your own RDBMS with an application-tuned replication and sharding scheme, so you'll want to hire or train at that point -- but there's huge huge room to play within the economic and technical characteristics of modern managed cloud RDBMS-as-a-service for any app that has a hint of a business model, given that we measure the cost of such services in cents per billion unit-hours.
But that's not a query I ever need. I load the user's document, which contains an array of slide (embedded documents). There is no relation between documents & slides, slides are part of the document.
But take it one step further & imagine you have a variety of widgets that can be placed on slides, with no common schema between them. Your choice is model out 100s of tables, use EAV, or go noSQL. We did the latter.
> would make me itching for indices and foreign constraints
Mongo has indexes. Regarding foreign keys it does not have that, but usually you can just embed the document in all the places you need it (de-normalization). Which can be a problem, so if you don't want de-normalization you should definitely use a SQL DB, not mongo.
> Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images"
Mongo has aggregation which has fortunately been able to perform all the reporting I've needed so far. I'm sure postgres is nice as well.
Because MongoDB stores documents in JSON form. It's the easiest data structure, easiest to transform in whatever language you're using, it's fast, and its scalable.
163 comments
[ 3.1 ms ] story [ 230 ms ] threadTo put it another way, RethinkDB did extremely well on Hacker News. Twitter didn't, if you remember all the negative posts (and still went public). There is little relation between success on Hacker News and company success.
To be clear, I've only ever run it as support for something else that depended on it. It just isn't well-suited to my projects and it wasn't interesting enough to invest much more than noodling a bit with it. I'm sure it is a fine product that turned in to a victim of its own hype.
It does serve as a decent example for the next generation of developers about how flavor-of-the-month trends in tech make people do stupid things, like decide they don't actually need ACID for handling money.
It's in vogue to hate on PHP especially here on HN. Language hate based on some sort of popularity contest is silly and language snobbery wont be tolerated at any company worth its salt (read: NOT most startups)
If you think PHP is looked down on by those who care about language features because of a popularity context, rather than because of asinine language decisions and limited support for modern programming paradigms, I think it's you who is missing it. Since any program can technically be written in any (Turing Complete) language, the fact that some big sites were made to work with PHP also sort of misses the point. When most of the big "success stories" adopted PHP, options for web-friendly open source languages were much thinner. Now if you look at the new development projects they highlight, you don't see PHP come up that much. Facebook, for example, is prominently using Haskell. I don't know anyone who both programs in Haskell and is willing to touch PHP.
Of course, many thoughtful criticisms of PHP have been discussed on HN. e.g. https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ & http://poincare101.blogspot.com/2011/11/php-sucks-but-some-f....
Still, that's not going to do much to change public perception, and I guess that's somewhat fair. PHP has earned it's reputation, even if many (but not all) criticism are now largely dated.
I'm just happy I switched to working on other languages, aside from the occasional Laravel-based side projects.
Is public perception the reason you are happy to be mostly using something else now, though?
I really enjoy some of the more flexible syntax and design choices present in some other languages. It also help that various job requirements over the year have required that I build a broader skillset. Finally, the growth I've experienced by stepping outside my comfort zone the past few years is probably the best benefit, which is a good experience regardless of which language(s) you start with.
In the end, however, as we both know, most decisions in software development are an exercise in managing tradeoffs. I guess the core of my argument is that, in the past few years, PHP has done a great job at improving its balance of tradeoffs.
I was curious and wanted to see if that is the case. So I checked for the most basic thing. The arrow notation for lambdas. Surely they must've added that by now. Surely they must've added that before Java did. Nope, still RFC. Now I wonder if it's idiomatic to pass those "long form" lambdas around when writing in functional style or is it something nobody does. In any event this little discovery makes me think parent's sentiment seems to be still justified.
Personally, having had to write a couple of Mediawiki plugins and an extension for a rubbish CMS called TYPO3, my experience so far is that the PHP hate is very much justified.
Seriously, the biggest problem with php is that fundamentally it's a templating language. One thing I've seen with templating libraries in general, is that they are meant to take a template and a data structure and display output based on these 2, but what ends up happening is they end up converging on turing completeness. It starts with the need for some kind of looping structure in the template. And then someone needs some random logic that is easier to do in the template than before the model, and so another feature gets added to the templating engine. And so it goes, things get kind of bolted on, without a grand vision, and you end up with an API that is inconsistent, where the order of inputs is never the same (is the first parameter haystack or needle??).
Also, the basic shape of the code that tends to get written in PHP is heavily influenced by it's being a templating language. PHP applications tend to end up having a lot of code in the template, that should be done before rendering the model. Rather than a sane 2 pass system, where all the data is gathered and the logic handled to produce a simple data structure that is passed to a very simple template, you end up with crazy monstrosities where there's opening and closing <?php> tags everywhere. And you render part of the page, then grab some more data, then render some more page, then maybe do a curl to a 3rd party vendor, then render some more of the page, and it's actually very hard to keep switching mental state to match the context of the code you're looking at.
Anyhoooo, you're getting downvoted because there's lots of literature on why PHP is a terrible language. I'm only replying because it's sort of cathartic, since I maintain a wordpress codebase, and any time I have to interact with core, instead of being able to sanely intercept the underlying data, modify it then pass it along in one place, I have to constantly add lots of random hooks in lots of random places to catch the data just before it's being rendered. It's a total mess, and far more complicated than it needs to be (though there is some debate whether this style made it more approachable from a novice plugin development perspective).
By that time I was doing most of my coding in Ruby and Python, but this businesslike approach to PHP made me realize this is a place where people really get their hands dirty and build things, rather than care about fads.
Things have really changed, the tone & level of discussion here are now very different. Seems like PHP is kind of a canary in the coal mine in that regard.
It's more like those vast parts of the internet are working despite PHP. Why do you think Windows is powering vast numbers of desktops? It's not for its technical excellency. ;)
So, it started as a shitshow on the back-end and people who had to maintain systems based on it hated it, but it got popular because it was easy to get simple systems up and running with. Then, over time, it slowly became less awful as it was substantially rewritten and large portions of it redesigned. I'd say PHP is a very, very good comparison.
Reminds me of: https://slack.engineering/taking-php-seriously-cf7a60065329
https://news.ycombinator.com/item?id=14659729
You make that sound like its not a big deal vs vanilla php.
The choice was narrower then. Ruby and Python, while already active, did not have nice web and DB frameworks (both RoR and Django started in 2005). Java was also much less mature, and even more wordy.
PHP, on the other had, was easy to get from any shared hoster.
OSM started in 2004, but the editing API was ported to Rails in 2007ish.
Downloads can be repeats, no? How to discern real number of productive installs?
Skills aren't objective, are they?
Revenue, great...but they won't disclose it.
For a long time all I had ever done was relational, but then my company started using Azure table storage for a lot of projects and I became familiar with other models. Then I discovered Mongo while catching up on Angular, and enjoyed using it with the mostly small apps and prototypes I build. I don't know it well enough to try and support it's use on a large scale. After seeing how poorly designed the data is for so many companies out there, it's not a surprise to hear poor choices were made (and probably some poor execution).
Current Indeed.com Job Trends data actually show better numbers than the 2102 stats, but nonetheless show MongoDB jobs in decline.
https://www.indeed.com/jobtrends/q-MongoDB.html
10gen is not a company dedicated to databases, at least not if the initial code quality of MongoDB is a signal (and it should be, since if you know a thing or two about storage you wouldn't take the compromises it initially did)
Unfortunately, many fresh graduates with no experience in storage decided to evaluate the technology based on its very visible merits, without considering that scalability is not really a top priority even in companies that have to "scale", and that reliability trumps every other metric.
Mongo has failed spectacularly in being a reliable data store, and its scalability is mediocre, given that it's a lot more about query planning, analytics and optimization than having out-of-the-box sharding.
This is one of those areas having some seniority does wonders. Storage concepts are not easy to grasp, and evaluating a database's reliability takes much longer than most product cycles.
And you might want to look at how SalesForce has implemented a highly scalable architecture on top of Oracle RDBMS servers. That includes a document store as well as a dynamic denormalized schema layer that works real well for reporting.
Heck, I've implemented a persistent memcache server on top of SQLITE and it scaled pretty well because modern servers and network infrastructure just happen to do most of the work needed.
It is not easy to learn PostgreSQL well enough to evaluate it, but more and more people are doing so and finding that it is the ideal open source persistence solution for the vast majority of business uses.
I'm not insulting Postgres (actually a big fan), but Postgres added JSON support after. By that metric, document store in Postgres is less tried-and-true than MongoDB. Does that mean it's the wrong choice? No. Age of product is not always a good qualifier.
Whatever else you might say about Postgres, that community cares about the integrity of its users data, over pretty much every other consideration. I don't think the same can be said of Mongo, or any other "NoSQL" anything.
I feel pretty good about DynamoDB, but the development model WRT to indexes and throughput is what keeps me away.
Except it's not and age of the product is a bad qualifier.
The MongoDB hate hype train is even worse than the MongoDB hype train ever was.
Unstructured documents are very rare. Querying arbitrarily against them even more so. For the fields that do need querying in a document, put them apart and add a few indexes and you're good. That's been my experience and I've been around a while.
And if you do not want to commit to either SQL or NoSQL, the PostgreSQL has you covered. You can put a JSONB column in any of your tables to handle the part of your data that is schemaless, and you can still index the data in that JSONB column.
There are a very few applications that need distributed scalability beyond what Oracle or PostgreSQL can provide, and those ones need NoSQL. They also need highly skilled competent developers who are capable of reimplementing some needed functionality of RDBMSes at scale.
There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
You also do have to think at least a little about your JSON structure given you can't really use an ORM to query JSONB, and the syntax for JSONB is a bit awkward at times.
It's much much harder to design a reliable engine than a new datatype.
> There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
Yes they are, but chances are you don't have that need. You're not Google or Facebook.
You also picked up a really bad database to talk about scalability[1]
[1] https://www.datastax.com/wp-content/themes/datastax-2014-08/...
SQL is a lot for newcomers to grasp on top of learning the rest of their stack. The appeal of "no schema" is most likely what gets them as it seems easier than SQL. Simple things at first seem daunting like: constraints, joins, and indexes. What they fail to see is that doing these database level things in the application layer becomes the final level hell and you get to live it.
People also undervalue their data and it's integrity.
I personally do not like mongo and see zero reason for it except perhaps in some very niche use cases and even then, I'd still choose another document store over it.
I'd say don't worry: I almost never use anything I learned in school.
The Java was to shallow as was lots of other stuff.
School is just there to convince your first few employers so you can het a chance to see how it is actually done.
Oh, and to give you the credentials to stop annoying sales guys that are about to convince your boss that "you need Active Directory because your network is slow". (Guys company had botched a certain server install and tried to solve it by selling even mire stuff. )
Red brick Russell Group British university, not prestigious.
The classes were not mandatory so you could technically graduate with CS without knowing anything about databases.
I did. I wouldn't use it again. But reliability just wasn't an issue. There was a time when there were insane defaults that made this an issue, but that's a long time ago.
Channel: chat.memsql.com My UN: eklhad
This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex.
No wonder the whole javascript stack is a complete and utter mess!
However, most people think NoSQL as NotRelational - when there is no ACID guarantee. They are choosing availability over consistency.
I'm pretty sure that's wrong and it always meant not to use the relational model; it was only “NoSQL” because it's short, easily memorable, and there's a percieved (though not exactly true) equivalence between SQL-based and relational databases.
http://jimgray.azurewebsites.net/papers/thetransactionconcep...
TBH, I'd rather write SQL directly rather than use an ORM though. I never want to use an ORM ever again. At least that's one problem that MongoDB solved.
I will never understand this attitude.
Why did you think you can only use SQL with an ORM? And if you didn't think that, then what was your point?
When it comes to Mongo, I think it's nice to have the DB schema documented somewhere but you don't really need to have it enforced with Mongoose... It introduces more problems than solutions.
With Mongoose, you have to run migrations on your data more often which slows you down. Though it might make sense for bigger companies.
1. Architecturally is was very familiar to traditional relational systems like MySQL. Meaning classical master/slave, sharding and absolute consistency via locks.
2. Mongo's rise coincided with the rise of NodeJS and continued on the success of Ruby on Rails.
3. Mongo has done a phenomenally fantastic job attracting developers with exceptional documentation and getting started materials.
These all combined into a perfect storm attracting legions of developers. Note that these things have absolutely nothing to do with the technical capabilities of Mongo as an actual database.
In point #2 substitute PHP for Node and RoR.
In point #3 just change the name from Mongo to MySQL. MySQL AB was brilliant at getting MySQL into Linux distros. When you came to need a DBMS to go along with PHP chances were you could just type 'mysql' and you would connect to a live DBMS instance on the host.
The 10gen team also imitated the MySQL playbook explicitly with built-in replication and an early lack of focus on ACID properties.
When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We wouldn't have to write SQL anymore (which was actually funny to watch one of our engineers try to unlearn table denormalization).
At the end of the day, it was hype. I had to stay up during multiple 2012 debates for 5 hours and flip the Mongo databases because they kept crashing. The shard key they set up for us was slightly out of balance so the data distribution was 30-30-40 and every so often, that server processing 40% of our data would jump to 50% and then knock that server offline leaving the other 2 to split 50% of the work... knocking them offline. There were also tons of replication problems all traced to write locking. At the end of the day 10gen solved one problem, but our company traded that solution for other problems.
That experience taught me that you really need to understand what you're trying to solve before picking a database. Mongo is great for some things and terrible for others. Knowing what I know now, I would have probably chosen Kafka.
Going from sql to nosql would be denormalizing data not normalizing it so, the engineer would be unlearning table normalization.
http://bit.ly/2uP3Obi
In order to leverage the multiple CPUs in a box you'd need to run multiple instances of mongo on the same machine and we designed a whole system around sandboxing instances.
But then you'd run into memory allocation issues and how badly it would handle indexing and handle running out of memory, which mostly involved crashing.
We ended up having to run about 100 bare-metal servers to support what would probably have been a level of queries about 10 PostgreSQL boxes could handle easily.
Truly a terrible DB.
It's like complaining getting graphic cards working because of drivers is difficult on Ubuntu, based on experience with linux in 2005.
I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days.
A lot of software fails, companies close, while some succeed. Some software that has been hyped up ended up not succeeding, but it doesn't matter. One learns from others' mistakes and keeps growing.
You would be correct, as the mountain of ignorance about MongoDB on display in this and all HN threads on MongoDB attest to.
And at the time, most of the mindshare is/was with mongodb for document-oriented technology.
My favorite recent one was a company that had to persist their Mongo database into AzureSQL (nightly job) so they could then push it to Good Data for analytics. They could have very easily used SQL Server and pushed directly from there to Good Data. And yes the company admitted that was the case.
Also the table is misleading, PostgreSQL replaced Oracle, Mongo replaced Oracle Coherence[1].
[1] https://en.wikipedia.org/wiki/Oracle_Coherence
Mongo did replace Oracle product but it was Oracle's Coherence[1]. There wasn't a concern of Mongo loosing data, because the data could be reloaded again from Postgres if it got lost.
I also heard that they were considering replacing Mongo with Postgres as well, but I do not know where this went.
I think 10th Gen is just good at generating hype. The migration was not because Mongo was better but because Oracle was expensive and not that much better from Open Source alternatives.
[1] https://en.wikipedia.org/wiki/Oracle_Coherence
It was a perfect mixture of this unfounded fear with an advertised simple solution. Add the idea that there is no need for a DBA, encourage developers to see Mongo as a magic data storage and nothing more, and you have the perfect product for those who don't feel the need to look deeper.
Startups seem more optimized to fast iteration and quick development. It seems to me that they chose Mongo because they prioritized iteration and ease-of-development over the hard-nosed style needed to use a relational database well.
Maybe it's an age thing (or the product of "bootcamps"), but I don't think that's true. I'm 40, and when I was learning, you connected to and wrote SQL against the database. These days I still spend a significant amount of time writing SQL, both for ad hoc queries and for complex queries / performance optimization.
> Add the idea that there is no need for a DBA
With many cloud database offerings this is essentially true. As for things like indexes, etc... developers targeting Mongo need these as well.
> encourage developers to see Mongo as a magic data storage and nothing more
In modern frameworks, this tends to be true no matter what the data store is.
MongoDB feels like a different product to what it was when the hype was high. The company, leadership and engineering quality certainly have changed.
Someone else commented about the HN bias, which we can't deny exists.
I've been running an 1-member replica for as long as I could figure out how replicas work. So I've had my Mongo database running for the last 5 years. It's met the sweet spot that I was looking for when I knew less than what I do today.
Even if one were to rubbish the product, it still has value as the company seems to be doing well (even with the negative sentiment and press). Mongo sometimes feels like how "corporates" (at least in the recent past) feel about open-source. Dealing with our SAS account at work, the sentiment used to be that open-source was a waste of our time. Open-source is brittle and slow, rather use Oracle, etc. SAP HANA will change your life, in-memory is hard, our "appliances" run on unsecular hardware ...
Those sentiments have changed over time, so we shouldn't eternally write things off if they're healthily maintained.
I look forward to mixing SQL and NoSQL well into my future. DocumentDB wouldn't be what it is today if there was no Mongo . The SQL vs NoSQL flame wars get us nowhere to be frank.
I can read their feature page but as someone who isn't an expert, I'm not sure what's marketing hype and what's genuine advantage.
Incidentally, http://www.lmfdb.org uses MongoDB (since maybe 2011) pretty heavily, as a result of me convincing them to do so after I read too much about MongoDB back then. It’s been painful for them, but it does still work well...
The mailing list is also nearly dead with only a handful of posts total during 2017: https://groups.google.com/forum/m/#!forum/rethinkdb
RethinkDB did a lot of things right from an engineering perspective, but they were too late to market to compete with MonogDB; they burned through their cash, then went out of business, and now the devs work at various other companies (e.g., Stripe). RethinkDB is not a viable choice today.
I personally put enormous effort into trying to use RethinkDB (and Cassandra, and also MongoDB); now I very happily use PostgreSQL. just like I did back in the 1999, and I’m very happy with PostgreSQL.
I wish someone would collect all the hyped up tech blog posts so that we always have a reference to be able to say, "see what hype does?" Sadly, I think it would only increase the number of devs saying, "Not this time, it's different."
Relational db's are hard. You have schemas and query planners and the truth is, you have to take care of every single query. Every query counts.
The same query can take 30 seconds, or 30 milliseconds to run, depending on it using an index or not. You think it's ok for it to take 2 or 3 seconds, until it hits production and halts the server.
Mongo doesn't have any of that. Just load and dump json back and forth, which is perfect for fast prototyping.
Who cares about data integrity or technical debt?
Mongo has query planners, the same thinking you do in SQL applies to Mongo, the edge cases are different. For example, you can't have composite indices on multiple arrays. The same principles we consider when tuning our RDB apply to Mongo, normalisation comes at different forms/levels. The same applies to denormalisation. Just because there are/were "no joins" didn't mean that everything goes into a single collection.
I run some Mongo deployments, and I take as much care in understanding performance, as I do with our SQL deployments. Databases are hard to those who are unwilling to open up the manual and read. It's not only SQL.
I don't think there's anything that should immediately preclude the use of MongoDB in a production environment. I personally wouldn't use older versions of it as a single source of truth, but there are definitely some great use cases for it, such as allowing developers who only know JavaScript to write non-critical backend services.
People seem to forget that database consistency is not binary, but residing on a spectrum. Like any other trade-offs, you should carefully measure the costs and benefits for your particular situation before making a decision.
Since Relational Algebra was invented in 1970 we were unable to find a better representation of a relational data, there were many attempts but so far they failed (except for very specialized uses). So I'm inclined to believe it's here to stay.
JSON is basically just step backwards back to the XML Database[1] and Object Oriented Database[2]. It starts falling apart as soon as your data becomes more complex (you have relations, or need to query it by something else than primary key). Having said that, I'm glad that relational databases added support to it, it perhaps might be useful in early stages when things are in flux.
[1] https://en.wikipedia.org/wiki/XML_database
[2] https://en.wikipedia.org/wiki/Database_model#Object-oriented...
If it is all in mongo? Welp. How is your python? "See there is this key, and you need to sum this -- OH and remember that this does.. Hmm, "
Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over.
With something like mysql I have to learn about & choose between row based replication or statement based replication. I have to choose between binary logging or global transaction ID logging. With Mongo I just call rs.init() and rs.add() and it "just works".
Perhaps I'm just not hitting the level of traffic where the problems begin to surface, or maybe postgres would be better... but its been great so far. No complaints. The only issue was de-normalized data getting out of sync due to bad coding in our app, which i do not blame Mongo for (and was simple to fix).
An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching for indices and foreign constraints across what definitely feel like relations to me.
And for the non-relational content blobs, Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images", "who are the top 10 users in terms of edits to content tree items, and what subscription level do they have" pretty neat and fast.
If you're in early stages such that you don't want to hire or acquire DBA skills, you're probably well served by offloading replication/backup/verification (which are all very possible with RDBMS) to e.g. AWS's RDS product.
HA is important, is covered by replication+failover. Periodic backups are important, are covered by a export+rotate schedule. Point-in-time restores are important, are covered by storing replication logs. All are covered by a managed RDBMS, so you need to know what they are and how they work, but not implement or manage them. Most of the NoSQL deployments I've seen just rely on the out-of-the-box replication, and get HA but that's it. The engineers I've seen that don't want to understand the simple high-level ramifications-to-developers of RDBMS replication don't want to understand ramifications-to-developers about their NoSQL replication either, and they take a risk of encountering those ramifications needlessly unforeseen.
As the data grows, yeah, maybe you'll eventually hit technical or economical limits that mean you'll want to manage your own RDBMS with an application-tuned replication and sharding scheme, so you'll want to hire or train at that point -- but there's huge huge room to play within the economic and technical characteristics of modern managed cloud RDBMS-as-a-service for any app that has a hint of a business model, given that we measure the cost of such services in cents per billion unit-hours.
But that's not a query I ever need. I load the user's document, which contains an array of slide (embedded documents). There is no relation between documents & slides, slides are part of the document.
But take it one step further & imagine you have a variety of widgets that can be placed on slides, with no common schema between them. Your choice is model out 100s of tables, use EAV, or go noSQL. We did the latter.
> would make me itching for indices and foreign constraints
Mongo has indexes. Regarding foreign keys it does not have that, but usually you can just embed the document in all the places you need it (de-normalization). Which can be a problem, so if you don't want de-normalization you should definitely use a SQL DB, not mongo.
> Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images"
Mongo has aggregation which has fortunately been able to perform all the reporting I've needed so far. I'm sure postgres is nice as well.
> Why do so many developers use map/reduce instead of the database for select, pluck, math?
> Why do so many developers use length instead of counting sql?
The learning curve was easy, it was very compatible with javascript, it was very easy to install, it had similar features to sql lite.