Looking for a technical debate: Why MOT use Redis as a primary DB?
Gasoline cars vs. Electric cars We all know that you can use an electric battery to run a car.
But the thing is, when it comes to a gasoline car, even though it does literally have a battery, it’s not used for running the car. It uses batteries for starting the cars (generating an electric spark to light up the gas), A/C, audio systems, lights, sensors, locks, and so on but not for running the car. Instead, it relies on an internal combustion engine (ICE) to run the car.
It turns out, ICE cars are highly inefficient. Only 16% to 25% of the power that’s generated actually makes it into the wheels. On the other hand, electric vehicles provide about 90% power to the wheels! Further, EVs also have major and additional advantages when it comes to the environment, repair costs, and so on.
If you are looking from the First Principles, even though virtually most cars that are built even today are gasoline cars, the fundamental truth is that they use an inefficient system.
Now if you look at an electric car, it takes advantage of this inefficiency to build a new type of car. In this case, it simply gets rid of the complex and highly inefficient engine and replaces it with a large battery and a motor to directly spin the wheels.
------------- Now coming to the Databases...
In the traditional architecture, you have a primary database (Postgres, Mongo, etc.) and a secondary database, a.k.a, cache (e.g. Redis or Memcache). The primary DB is used to store all the data and support CRUD operations. The caching DB is used for caching, session storage, rate-limiting, IP-whitelisting, Pub/Sub, queuing, and many other things.
And if you think about it, when there is a cache-hit, we are practically using the secondary DB for part of the CRUD operations, but still not fully utilizing it as a primary database.
Does this remind you of the issue with gasoline cars? Just like they literally carry a battery to power numerous things except moving the car, the traditional architectures use things like Redis for everything else except as the main DB.
Do you see the similarities?
What if we use the First Principles thinking to do what the electric car did? Similar to how EVs got rid of the engine, what if we get rid of the slow and inefficient primary database and simply use the cache DB as the main database?
https://redislabs.com/blog/dbless-architecture-and-why-its-the-future/
14 comments
[ 3.5 ms ] story [ 35.3 ms ] threaddata integrity without ACID?
complex queries?
For failure use sentinel or cluster - works great
BTW, you're way off the mark with the numbers in your car analogy! I know the precise numbers are not the point, but it's really not 90% vs 25% efficiency. The oft-quoted figure of 25% efficiency refers to thermal efficiency. It's a measure not of how much power is wasted by the car, but how much of the energy in the fuel is turned into useful work. To compare electric cars, you have to consider how the electricity is generated. It's mostly generated by burning fossil fuels. It's more efficient than petrol cars because it's done at such a large scale, but it's not 90%. It's more like 40-50% afaik.
The car efficiency numbers came from Inside EVs as the article points out. I think they did a pretty thorough job. https://insideevs.com/features/392202/ice-vs-ev-inefficient-...
That's not really much of a "debate", but then again neither is "you totally don't need that feature we don't implement" from the marketing department :P
https://redislabs.com/blog/you-dont-need-transaction-rollbac...
I think the blog talks about that in two sections. "First reason to use rollbacks: concurrency" and "Second reason to use rollbacks: leveraging index constraints" says why it's different in Redis.
Going back to the car analogy, if you are an electric vehicle, some things are completely obsolete when compared with gasoline vehicle.
https://kristoff.it/blog/addio-redis/
I stand by my words and in fact a good chunk of the company was not happy with my reasoning in that post (it's "better" to be coy than upfront about these things, according to some schools of thought), that said: you don't need rollbacks in Redis, whoever argues the opposite has never spent enough time learning how to use it.
An RDMS is a centralized, consistent source of truth, with more normalized data, that can answer most questions with reasonable performance.
When you remove the RDMS, what happens?
- You lose the ability to answer ad-hoc relational questions (what stuff does the user own? And in those things, which of them is located in Chicago?). This means building a new feature means building a new "cache" aka a brand new data structure _just for this one use case_ that might be a one-off
- You lose a centralized PoV on your data consistency. One view of the cache says the user's item is in Chicago. Another says its en route to LA from Chicago. How do you resolve these conflicts? Are you going to build your own consistency systems? Based on what exactly? Now you've essentially built a new kind of distributed database.
- Much of the time we don't need caches. If you always have to bust your cache, the cost of constantly rebuilding a cache, just to throw it away, can greatly exceed any value from that cache. Most people can just read from the RDMS and get what they need for 90% of the use cases in most apps.
BTW I used Redis as a primary DB for a few years for Quepid. You can read the full story here
https://www.slideshare.net/AllThingsOpen/stop-worrying-love-...
Long story short, it was fine, but Redis didn't allow for much structure. There was a lot of "logical" relational data modeled very awkwardly. It made it hard to extend beyond the original data model.