Since the consumers of Redis are in Heroku (or similar), it seems that you have little options to restrict the access properly (note that I'm not a security expert!).
I'd love to use Redis with DotCloud or Heroku but I'm fairly concerned after reading the documentation, and so far I avoided it outside closed LAN setups.
Thanks to anyone able to provide some insights on this!
From what I've seen some providers spin up your own EC2 instance and others share EC2 instances between customers. The first way seems more secure, but I assume there is a way to make the second secure as well. It's a great question to be asking, I'll check around.
I'm not an EC2 expert, but I seem to recall there are security groups that you can define etc.
If you rely on a PaaS like Heroku or DotCloud, your consumers instances could change anytime, so there is probably no easy/reliable way (apart if they support the notion of security group in some way) to restrict the connections on the EC2 running Redis.
Just raw thoughts really - but I'd love to safely use Redis in the cloud, mixing Heroku/DotCloud or similar on one side, on RedIsGreen/RedisToGo on the other side!
I don't know any of these providers or how they operate, but since redis usually uses TCP to communicate it shouldn't be too difficult. You can set up different instances of redis, each listening on a different port and serving only one customer. Using SSL or SSH tunnelling can encrypt the connection between the client and the redis instance. Adding IP filtering to prevent access from only the client's server can also be added for extra layer of security.
Performance, which is very important in those circumstances might be impacted, but it might already be affected by at least some latency, and the extra security layer shouldn't degrade it that much more. In fact, SSH can add compression that can speed up things such as replication (see http://3scale.github.com/2012/07/25/fun-with-redis-replicati...). Not sure how much this helps on regular client-server communication though (sorry for the slight digression into performance)
My understanding is that restricting the access is the tricky issue here.
When you take a redis instance provided by a given host (redistogo etc), and want to make sure only front/web instances from another host (dotcloud, heroku), unfortunately to my knowledge you cannot restrict IP easily (or secure with more useful means) etc, since you don't control the machines as much as in a LAN setup.
Using SSL is definitely a plus, but being accessible from the whole internet, as I seem to recall Redis is very fast, it is also easier to brute force, so one has to be extra careful to pick a very strong password.
I'd sleep better at night if the connection was only possible from the front/web machines, like in a close LAN, and I cannot see how to do that properly when the app and Redis are not hosted at the same provider.
You can actually break down the security issue into 2 distinct problems:
Problem 1: unauthorized access.
If an unauthorized client can issue redis commands, you're screwed. One way to prevent that is to block all network trafic to redis, except for a whitelist of trusted clients. As explained above this is not possible on a PaaS because a) your application containers share source IPs with other, untrusted containers, and b) those source IPs may change at any time as your containers are moved around.
However, there's another way: using a master password. With this optional feature, Redis can be configured to authenticate all queries with a single statically-defined password. You have to use it well: specifically, make it long enough to sustain brute-force attacks, and be careful where you store it. Luckily, dotCloud (and presumably other redis providers) implements these best practices for you so you can benefit from the security.
Problem 2: trafic interception.
Redis does not support encryption, so if Alice can eavesdrop on the network, she can intercept and possibly forge Redis commands. If you can't trust EC2's network, this means you can't use Redis on EC2 at all (independently of problem 1), unless you implement end-to-end SSL encryption with something like stunnel. However, it's more likely that you, like the majority of EC2 customers, already have to trust their network anyway. In that case, you're probably Ok using Redis on EC2 and therefore on dotCloud.
So, to summarize:
- A well-engineered PaaS (and specifically dotCloud) protects you from unauthorized access
- If you can tolerate the risk that your public cloud provider (or somebody who hacked them) intercepting network trafic, you can use Redis on a PaaS. Otherwise you need to implement your own service with end-to-end encryption (keeping in mind that your public cloud provider also has access to encryption keys stored on disk and memory)
Stunnel (which uses SSL) allows you to use client-side certificate to authenticate the client side of the connection. Similarly you can define SSH tunnels using public-key authentication. The same way github authenticates access to git repositories. You can also use VPN connections like OpenVPN etc.
So even if you can't restrict by source IP, you should be able to add another layer of security wrapping redis, providing both authentication and traffic protection.
That's correct. The only downside of client certificates is that they can be a PITA for developers, especially if you find yourself frequently spinning up new dev and staging environments. Also, on a PaaS this might force you to check-in your certificate alongside your code in order to deploy it. On dotCloud you can avoid this with the rsync upload method (which is now the default in 0.9).
There's also security through obscurity. If you use "requirepass" AND rename the "auth" command to something only the client knows (yes...you can rename commands in redis), then you could potentially "partially" secure the instance even more (although someone can basically DOS your server attempting to "auth").
With Heroku, they give you their user id so you can tell EC2 to only allow connections from the heroku servers. It limits it a bit more than other PaaS, but it is still not pretty secure.
Interesting comparision. An additional factor to take into account is whether the instances are 32 bit or 64 bit - that will affect the memory usage quite significantly.
I have used RedisToGo, and it was one of the worst products I've ever had the displeasure of dealing with.
I lost contact with the instance associated with my staging server one morning and filed a support ticket immediately. It was almost 4 days before someone responded, even after daily messages to their support team. After requesting my account information, I was simply sent a link to the "How to Connect to your Redis Instance" help page and the ticket was closed. I reopened the ticket and explained that the connection was not the problem, I was then told that there was no problem and the ticket was again closed. Other than in my initial ticket, I was never again asked to explain my issue, or offered any real support. I determined the service was no longer worth my time and booted up an EC2 instance.
Fast-forward about two months. I now receive an email from RedisToGo: "We are planning some scheduled down time that affects all users on server X because of system issues." The message then proceeded to describe the symptoms I had explained to them months before.
I sent their support team a very polite email requesting an apology for the way my issue was handled. A day later, I received a half-hearted apology and 1 month credit for some advanced feature they offered. I never cashed it in.
Never, ever, use RedisToGo if you need a reliable Redis instance.
15 comments
[ 4.6 ms ] story [ 32.8 ms ] threadThis http://redis.io/topics/security seems to indicate you have to make sure you're in a safe zone to use Redis.
Since the consumers of Redis are in Heroku (or similar), it seems that you have little options to restrict the access properly (note that I'm not a security expert!).
I'd love to use Redis with DotCloud or Heroku but I'm fairly concerned after reading the documentation, and so far I avoided it outside closed LAN setups.
Thanks to anyone able to provide some insights on this!
If you rely on a PaaS like Heroku or DotCloud, your consumers instances could change anytime, so there is probably no easy/reliable way (apart if they support the notion of security group in some way) to restrict the connections on the EC2 running Redis.
Just raw thoughts really - but I'd love to safely use Redis in the cloud, mixing Heroku/DotCloud or similar on one side, on RedIsGreen/RedisToGo on the other side!
Keep us posted on your findings :-)
Performance, which is very important in those circumstances might be impacted, but it might already be affected by at least some latency, and the extra security layer shouldn't degrade it that much more. In fact, SSH can add compression that can speed up things such as replication (see http://3scale.github.com/2012/07/25/fun-with-redis-replicati...). Not sure how much this helps on regular client-server communication though (sorry for the slight digression into performance)
When you take a redis instance provided by a given host (redistogo etc), and want to make sure only front/web instances from another host (dotcloud, heroku), unfortunately to my knowledge you cannot restrict IP easily (or secure with more useful means) etc, since you don't control the machines as much as in a LAN setup.
Using SSL is definitely a plus, but being accessible from the whole internet, as I seem to recall Redis is very fast, it is also easier to brute force, so one has to be extra careful to pick a very strong password.
I'd sleep better at night if the connection was only possible from the front/web machines, like in a close LAN, and I cannot see how to do that properly when the app and Redis are not hosted at the same provider.
You can actually break down the security issue into 2 distinct problems:
Problem 1: unauthorized access.
If an unauthorized client can issue redis commands, you're screwed. One way to prevent that is to block all network trafic to redis, except for a whitelist of trusted clients. As explained above this is not possible on a PaaS because a) your application containers share source IPs with other, untrusted containers, and b) those source IPs may change at any time as your containers are moved around.
However, there's another way: using a master password. With this optional feature, Redis can be configured to authenticate all queries with a single statically-defined password. You have to use it well: specifically, make it long enough to sustain brute-force attacks, and be careful where you store it. Luckily, dotCloud (and presumably other redis providers) implements these best practices for you so you can benefit from the security.
Problem 2: trafic interception.
Redis does not support encryption, so if Alice can eavesdrop on the network, she can intercept and possibly forge Redis commands. If you can't trust EC2's network, this means you can't use Redis on EC2 at all (independently of problem 1), unless you implement end-to-end SSL encryption with something like stunnel. However, it's more likely that you, like the majority of EC2 customers, already have to trust their network anyway. In that case, you're probably Ok using Redis on EC2 and therefore on dotCloud.
So, to summarize:
- A well-engineered PaaS (and specifically dotCloud) protects you from unauthorized access
- If you can tolerate the risk that your public cloud provider (or somebody who hacked them) intercepting network trafic, you can use Redis on a PaaS. Otherwise you need to implement your own service with end-to-end encryption (keeping in mind that your public cloud provider also has access to encryption keys stored on disk and memory)
So even if you can't restrict by source IP, you should be able to add another layer of security wrapping redis, providing both authentication and traffic protection.
See https://devcenter.heroku.com/articles/external-services for details...
I lost contact with the instance associated with my staging server one morning and filed a support ticket immediately. It was almost 4 days before someone responded, even after daily messages to their support team. After requesting my account information, I was simply sent a link to the "How to Connect to your Redis Instance" help page and the ticket was closed. I reopened the ticket and explained that the connection was not the problem, I was then told that there was no problem and the ticket was again closed. Other than in my initial ticket, I was never again asked to explain my issue, or offered any real support. I determined the service was no longer worth my time and booted up an EC2 instance.
Fast-forward about two months. I now receive an email from RedisToGo: "We are planning some scheduled down time that affects all users on server X because of system issues." The message then proceeded to describe the symptoms I had explained to them months before.
I sent their support team a very polite email requesting an apology for the way my issue was handled. A day later, I received a half-hearted apology and 1 month credit for some advanced feature they offered. I never cashed it in.
Never, ever, use RedisToGo if you need a reliable Redis instance.