I'm a .NET developer and my only experience with MongoDB was at a company where we spent more time fighting against MongoDB that just using it. We didn't have those problems at a later company where we used DynamoDB. Could be that the company who was using MongoDB was using it wrong or the DB team didn't really understand it (versus the AWS shop where the Dynamo guy knew the innards of the database by chapter and verse). Fair or not I've had had a "no, thank you" opinion of MongoDB since then.
>versus the AWS shop where the Dynamo guy knew the innards of the database by chapter and verse)
Definitely has a big impact on it. The angle at which people approach these databases is completely different from my experience. MongoDB is advertised with words like "developer friendly" and "just works", so people tend to just start it up and not read too much into it.
On the other hand DynamoDB was described to us as "monkey's paw", "Carefully read before use", "know what you're doing". And so we combed through first all the pitfall blogs, then AWS documentation, and then carefully tried out all the features we wanted to use and tested the scenarios separately.
DBs in general have a lot of unavoidable complexity swept under the rug. Like, I don't think most people realize Postgres and MySQL don't do full ACID transactions by default. But it's even worse how special DBs made for huge shardable data like Mongo market themselves as beginner-friendly.
Also the SQL timestamp vs timestamptz thing is just silly and avoidable. I want to know who I complain to for that.
How does he know how a black box database works inside and out?
That's an issue with dynamo. Sure you can memorize the API, but you have no idea what is actually going on and aws support will NEVER tell you what is happening.
Postgres, Cassandra, etc are source available and yes I have multiple times looked at Cassandra code to figure out what it was doing.
We use Mongo along Postgres and others at my workplace (backend is in .NET) and we don't have any issues with it. You just have to take into account its particularities as you have to do for any data store.
I like these kinds of blog posts. Not because he's ditching mongo, I like the detail around his thought process because there's an over-abundance of articles about "large company does X" but less of the hobbyist version of the same.
Not gonna lie, seeing him ditch Mongo for Postgres then say "API calls generally take 8 ms now, not 150 ms" made me a little happy. Mongo can make sense, but it seems like he was thoroughly on the hype train for it rather than actually needing it.
In the article, he mentioned that working in JSON seemed natural and comfortable. So after checking to be sure mongo was fairly mature and commonly used, went with it.
Postgres had good JSON support in 2019, and he later realized JSON wasn't actually needed. Hype train is what made MongoDB a default recommendation when it's really a more niche option.
>Finally, writing inserts and updates as JSON is convoluted. What should be a fairly simple operation like: "atomic increment this nested value, or insert if value is not there" was simply a disaster. For example, increment "api" by one, or insert "api":1 if necessary [...]
Maybe I'm missing something, but wouldn't `db.coll.update_one(search_query, {"$inc": {"visit_count.api": 1}})` do the trick?
Overall, I've been very happy with MongoDB (and their Atlas serverless offering). I would still default to recommending Postgres "by default", but for those use cases where MongoDB makes sense, it shines.
I think that MQL is very difficult to write because of its verbosity—it simply requires so much text. Even as someone who regularly wrote and debugged MQL queries, it is just so verbose, but essentially is just an AST structure and doesn’t need that much verbosity. I created a library that attempts to solve that friction, but I have no idea if it’s still being maintained by MongoDB: https://github.com/mongodb-labs/pymongoagg
It's definitely verbose, and working with it in Python is more annoying than in Javascript, due to the need to enclose all the operators in quotes. At the same time, I feel that the verbosity provides clarity, particularly in certain, more complex operations.
Seems like your library was last updated in April, though just basic maintenance. It looks pretty interesting! I might play around with it next weekend.
How can we know he used Mongo properly or not? Choosing Mongo because you don't know SQL doesn't seem valid. Likewise, choosing Mongo because "JSON is comfortably built into Python" doesn't seem valid.
Also, that's harder to do a proper Mongo implementation than to do a SQL implementation.
At a former workplace, when I stress tested the same API with different backend databases wired to it, Mongo was faster than Postgres. But we ended up using Postgres + Redis, because that combo was even faster than Mongo.
He did say he's not an expert. If anything it's meant as advice for others with small projects.
"I'm trying to stay humble. Mongo must be an incredibly big project with lots of nuance. I'm just a solo developer and absolutely not a database engineer. I've also never had the opportunity to work closely with a good database engineer. However I shouldn't be seeing improvements like this with default out of the box PostgreSQL compared to all the things I tried over the years to fix and tune Mongo."
But I would not assume that people in bigger companies are always experts in what they deal with either. It kinda matters how well something performs in non-ideal scenarios.
You don't. I don't. This is just my story where I'm clearly not a junior but had a ton of problems with mongo.
>Choosing Mongo because you don't know SQL doesn't seem valid
Valid is such a strange word here. People choose technologies based on their current skills all the time. If you've never encountered that kind of decision before then you need more experience.
34 comments
[ 3.1 ms ] story [ 64.8 ms ] thread>versus the AWS shop where the Dynamo guy knew the innards of the database by chapter and verse)
Definitely has a big impact on it. The angle at which people approach these databases is completely different from my experience. MongoDB is advertised with words like "developer friendly" and "just works", so people tend to just start it up and not read too much into it.
On the other hand DynamoDB was described to us as "monkey's paw", "Carefully read before use", "know what you're doing". And so we combed through first all the pitfall blogs, then AWS documentation, and then carefully tried out all the features we wanted to use and tested the scenarios separately.
Also the SQL timestamp vs timestamptz thing is just silly and avoidable. I want to know who I complain to for that.
That's an issue with dynamo. Sure you can memorize the API, but you have no idea what is actually going on and aws support will NEVER tell you what is happening.
Postgres, Cassandra, etc are source available and yes I have multiple times looked at Cassandra code to figure out what it was doing.
Sounds reasonable and not hype train driven
As opposed to?
You can use an SQL db engine as a kv-store, but you can probably make a more efficient kv-store if you do not have the SQL constraints.
(Oth SQL engines are pretty efficient for lots of use cases these days)
Another example where SQL may not be the right choice is time series or graph DBs.
Maybe I'm missing something, but wouldn't `db.coll.update_one(search_query, {"$inc": {"visit_count.api": 1}})` do the trick?
Overall, I've been very happy with MongoDB (and their Atlas serverless offering). I would still default to recommending Postgres "by default", but for those use cases where MongoDB makes sense, it shines.
I think that MQL is very difficult to write because of its verbosity—it simply requires so much text. Even as someone who regularly wrote and debugged MQL queries, it is just so verbose, but essentially is just an AST structure and doesn’t need that much verbosity. I created a library that attempts to solve that friction, but I have no idea if it’s still being maintained by MongoDB: https://github.com/mongodb-labs/pymongoagg
Seems like your library was last updated in April, though just basic maintenance. It looks pretty interesting! I might play around with it next weekend.
I always come back to Postgres.
Also, that's harder to do a proper Mongo implementation than to do a SQL implementation.
At a former workplace, when I stress tested the same API with different backend databases wired to it, Mongo was faster than Postgres. But we ended up using Postgres + Redis, because that combo was even faster than Mongo.
He has a smaller website and should be able to use mongo out of the box.
Complaining he didn't do indepth optimizations is ridiculous. He picked 2 database products and used out of the box configs.
Mongo should adjust their defaults if the defaults aren't using the product correctly
"I'm trying to stay humble. Mongo must be an incredibly big project with lots of nuance. I'm just a solo developer and absolutely not a database engineer. I've also never had the opportunity to work closely with a good database engineer. However I shouldn't be seeing improvements like this with default out of the box PostgreSQL compared to all the things I tried over the years to fix and tune Mongo."
But I would not assume that people in bigger companies are always experts in what they deal with either. It kinda matters how well something performs in non-ideal scenarios.
> How can we know he used Mongo properly or not?
You don't. I don't. This is just my story where I'm clearly not a junior but had a ton of problems with mongo.
>Choosing Mongo because you don't know SQL doesn't seem valid
Valid is such a strange word here. People choose technologies based on their current skills all the time. If you've never encountered that kind of decision before then you need more experience.