21 comments

[ 3.7 ms ] story [ 57.6 ms ] thread
Have to say consul is one of the most useful tools for service discovery i've ever used. Only feature I want is better support for multiple environments in the same cluster set.
When we used this, we used the first directory to be the env, so it looked like.

    /prod/...
    /stage/...
    /dev/...
    /feature-xyz/...
It's not elegant, but it made for really simple scripts that let us prepend /${env} to every key to access what we needed.

And of course, each service per env had different access tokens so apps in the stage env couldn't read prod keys.

A great release with lots of important bug fixes that were giving us issues.
Thanks for some great tools guys, and a shout out to Fabio another great tool built on top of Consul https://github.com/eBay/fabio
How does Fabio handle being a single point of failure?
This might be naiveté on my part, but I would assume you run multiple Fabio instances and treat it like you would any other LB.

Do you mean something else by your question?

Yes I meant something different. Generally with a load balancer the load balancer hosts that VIP(s) that your public facing DNS A records resolve to. In order to avoid a single point of failure you employ things like keepalived or ECMP/BGP etc.
The general setup seems to be that you have a public facing LB which just forwards all incoming traffic to any available fabio instance which handles the dynamic routing. fabio does not solve the public VIP problem for you.

Disclaimer: I'm the author.

You can run Fabio alongside the Consul agent on each node in your cluster and point all your apps to localhost:9999. That way you don't have to manage a central cluster of Fabio instances and if a node dies you just lose the Fabio instance on that host, which was only being used by that host.
Consul combined with consul-template for dynamic config changes in your environment is an extremely powerful solution which just works with minimum fuss.

Works well when deploying clusters such as zookeeper, allows all the nodes to discover each other automatically and work out their id number.

I am curious, why you are using both ZK and Consul as there is much overlap there.
There's lots of software you might operate that requires zookeeper: kafka, solrcloud, the hadoop ecosystem, etc.

Meanwhile, consul has lots of functionality built in that makes it a snap to use for service discovery within your existing infrastructure. You would have to build all this yourself on top of zookeeper, which just provides a distributed, consistent key-value store.

So it's easy to see why you could end up with both.

https://www.consul.io/intro/vs/zookeeper.html

Glad to see both etcd and consul support changing multiple keys atomically which I think is a critical feature.

Has someone recently compared the two in production?

Both projects have made some bigger changes in their Raft implementation since the last Jepsen test. I'd love for Kyle to re-do the test and also take into account the transaction feature.

Wish they would fix the issue with the watch triggers.
Has anyone upgraded from 0.6.x yet? Care to share your experience?
The new Lifeguard feature looks really neat. False monitoring alarms are a big issue. If they happen a lot you begin to not trust a real alarm and ignore them - the boy who cried wolf as the cliche goes.