HN Explorer

Ask HN: High Availability Redis?

8 points by modoc ↗ HN
Redis seems like a good fit for a portion of a larger web-app I'm working on. The one thing I haven't found is a good, simple, high availability/failover solution. Redis Cluster isn't out yet, and there doesn't seem to be any automated Master-Slave failover/recovery system, everything has to be done manually.

I'd think you could write a script to handle the Redis failover actions (promoting the Slave to Master, making sure the old Master comes back as a Slave if it comes back, etc...), tie it into heartbeat with a VIP, and have a very simple failover solution. But I can't find anyone who's done that, much less released the script they're using for the Redis piece.

So far the advice I've gotten is either: you don't need automated failover - just do it manually when you get paged, wait for Redis Cluster - it'll be magical, or write your own client - and have it handle failover and manage the Redis actions. None of these is viable imho.

What are you doing now to keep Redis from being a single point of failure in your infrastructure?

6 comments

[ 4.9 ms ] story [ 21.9 ms ] thread
latch
For the time being, your best bet is probably to rely on the handful of generic failover solution (keepalived, Linux-HA)...they are well documented and robust. You can connect to the slaves and issue a SLAVEOF command when your scripts pick a new master.

I know it isn't against the guidelines, but using HN to ask questions which are probably better suited on SO (or, in this case, the very lively redis IRC channel or google group) seems to be increasingly common. Also, doing a simple google search for "Redis automatic failover" returns a number of useful resources.

modoc
I appreciate your response, but basically you're saying there's nothing out there that handles the Redis side of failover, and I DO need to write custom scripts that handle all that, right? I can do that, but it seems odd to me that no one has done that and made them available already.

The reason I asked here, is I've seen lots of people posting about using Redis here, so I figured there would be some smart people with experience with the product.

Googling for that phrase (which I did before I posted here), gives me the same three answers I listed in my initial question, none of which are very satisfactory.

High ranking results:

http://www.quora.com/How-do-you-implement-automatic-failover... (Wait for Redis Cluster to be done)

http://groups.google.com/group/redis-db/browse_thread/thread... (Write custom client, and then a link to a script that 404s)

http://groups.google.com/group/redis-db/browse_thread/thread... (automatic failover is hard, just get paged and do it manually)

http://groups.google.com/group/redis-db/browse_thread/thread... (Write custom stuff and/or wait for Redis Cluster)

http://stackoverflow.com/questions/3097324/redis-handling-fa... (write a custom client)

And so on. Honestly none of that is useful. I can't be the only person out there who needs automatic failover for a production environment:( If I have to be the first person to write those scripts, that's fine, but it seems like someone must have done this before me.

scalingexperts
I've written a script to handle Redis failover and "promotion" from Master to Slave using Keepalived, but I haven't published it since it was never used in a production environment. It was actually written for Redis 1.02 (wrote it back in 2009).

I'll create a repository on GitHub for you to check out and let me know if it solves your problem. Please give me a couple hours because I need to eat some food like ASAP lol.

scalingexperts
Here's something, as I mentioned previously, it was written in 2009 for Redis 1.02 so it might not work as expected. Please feel free to fork, modify and improve.

https://github.com/alexwilliamsca/redis-failover

modoc
Thank you very very much!!
scalingexperts
You're welcome! I've updated it slightly to support the latest stable version of Redis. I hope it's helpful for you. Cheers!