HN Explorer

Can you replace Postgres with Redis (for some usecases)?

3 points by node-bayarea ↗ HN
Hi there, I'm looking for a technical discussion. I'm looking for technical debate and I know what MySQL/PG and Redis are capable of. What I'm looking for is a debate where you disagree that following. I want to start with an analogy. Gasoline cars vs. Electric cars We all know that you can use an electric battery to run a car.

But the thing is, when it comes to a gasoline car, even though it does literally have a battery, it’s not used for running the car. It uses batteries for starting the cars (generating an electric spark to light up the gas), A/C, audio systems, lights, sensors, locks, and so on but not for running the car. Instead, it relies on an internal combustion engine (ICE) to run the car.

It turns out, ICE cars are highly inefficient. Only 16% to 25% of the power that’s generated actually makes it into the wheels. On the other hand, electric vehicles provide about 90% power to the wheels! Further, EVs also have major and additional advantages when it comes to the environment, repair costs, and so on.

If you are looking from the First Principles, even though virtually most cars that are built even today are gasoline cars, the fundamental truth is that they use an inefficient system.

Now if you look at an electric car, it takes advantage of this inefficiency to build a new type of car. In this case, it simply gets rid of the complex and highly inefficient engine and replaces it with a large battery and a motor to directly spin the wheels.

------------- Now coming to the Databases...

In the traditional architecture, you have a primary database (Postgres, Mongo, etc.) and a secondary database, a.k.a, cache (e.g. Redis or Memcache). The primary DB is used to store all the data and support CRUD operations. The caching DB is used for caching, session storage, rate-limiting, IP-whitelisting, Pub/Sub, queuing, and many other things.

And if you think about it, when there is a cache-hit, we are practically using the secondary DB for part of the CRUD operations, but still not fully utilizing it as a primary database.

Does this remind you of the issue with gasoline cars? Just like they literally carry a battery to power numerous things except moving the car, the traditional architectures use things like Redis for everything else except as the main DB.

Do you see the similarities?

What if we use the First Principles thinking to do what the electric car did? Similar to how EVs got rid of the engine, what if we get rid of the slow and inefficient primary database and simply use the cache DB as the main database?

https://redislabs.com/blog/dbless-architecture-and-why-its-the-future/

12 comments

[ 6.6 ms ] story [ 41.2 ms ] thread
hardwaresofton
They’re different databases that do different things with different features and different ways of laying out data. Redis functions as a cache, not a relational database — it’s not merely about speed of response.

Do not use redis as your main database, it will require you to contort very painfully to so things that are normal to most apps. Maintaining Lua scripts instead of SQL queries is a bad idea.

If this is an ad for redis labs then it’s a really bad one.

node-bayarea
Can you give me a concrete example of were you'd contort if you were to use Redis versus Postgres? BTW, I'm just trying to really find an actual proof as opposed to mere anecdotal knowledge.
hardwaresofton
Any sufficiently relational query... Try and generate dynamic reports on data, or an aggregation on an aggregation.

The model subtly breaks even on simple questions like "I want to know all users who are currently logged in and the contents of their feed right now, with the last 5 messages they sent". You basically have to do multiple requests, MULTI, or write a lua script to get all this to happen in a transaction (so the state of the world is consistent). Such a request is trivial in relational databases, though performance of the query may not be.

hardwaresofton
They’re different databases that do different things with different features and different ways of laying out data. Redis functions as a cache, not a relational database — it’s not merely about speed of response.

Do not use redis as your main database, it will require you to contort very painfully to so things that are normal to most apps. Maintaining Lua scripts instead of SQL queries is a bad idea.

If this is an ad for redis labs then it’s a really bad one.

If anything you should be going the other way, postgres is rocksolid and you could literally run redis as a postgres extension if you wanted. It is the more flexible, battle proven tool.

node-bayarea
I understand that Postgres is an awesome DB. But my point is, why not use Redis especially when it comes with RedisJSON, RediSearch, RedisGraph, etc and gets most of the things done without any SQL. Why should we stick to the traditional approach? Is there any actual proof that Redis won't be able to handle it other than just guessing?
hardwaresofton
Because just like Postgres, there's a reason people don't use it for absolutely everything. Usually, purpose built tools will offer better features, performance, etc.

I am digging a company out of using Redis as a main database right now. They put everything in Redis, and the DB grew to be very large and they are paying a ton because they chose this. As far as the actual application logic it is using idiomatic redis, and it works but it is absolutely not optimal.

As far as I'm concerned the question is backwards, why would you want to use Redis for everything? Are you just trying to avoid SQL? SQL is basically the optimal solution (see the field of Relational Algebra) for querying relational data (and almost all data that matters is relational).

The boring "right" answer is use the right tool for the job. It's unlikely that RedisJSON is better (depends how you measure of course) than MongoDB/RethinkDB, RedisGraph better than Neo4J, RediSearch better than ElasticSearch/etc. If I wanted to pick a technology to go all in on, it definitely wouldn't be Redis -- it'd be Postgres (which has JSON support, Graph support, timeseries storage support, full text search built in), a fully open source product with a creator that hasn't left the project, with way more than one company offering tons of support and advanced behavior, etc.

BTW if you really like Redis, check out KeyDB[0]. KeyDB can fallback to FLASH storage so it might actually work in a large data case (I considered switching my client to it.

[0]: https://keydb.dev/

tarun_anand
There are lots of in memory Sql speaking stores like Memsql and voltdb. Won't they be a better fit than Redis in case you wanted the speed that comes with accessing data in memory?

What use case are you looking at?

node-bayarea
I'm looking at a retail store app. Redis comes with RedisJSON, Search and other modules to model it. So why not use it?
tarun_anand
Sure you can use it. It's just convenient to have tooling and libraries that speak SQL.

The impedance mismatch between application and data layer goes away.

If the primary benefit is in memory access you get that too.

shoo
Analogies can be a useful way to generate questions but dangerous when used to provide answers.

-- misremembered paraphrasing of someone else's snappy quote

So, a fun question could be: what are "first principles" for car engine design? what are "first principles" for data storage in distributed system design? are those two sets of first principles the same, or are they wildly different, with different goals and constraints?

i clicked through the link and was sad to see no clear articulation or evidence of any kind of "first principles" thinking. it would be genuinely interesting to see a solid analysis.

edit: aha, it seems like it i am remembering something Martin Fowler wrote once:

> [...] I'm very suspicious of using metaphors of other professions to reason about software development. [...] it all comes down to how you use the metaphor. Comparing to another activity is useful if it helps you formulate questions, it's dangerous when you use it to justify answers.

- https://martinfowler.com/bliki/MetaphoricQuestioning.html