Consistent hash ring Python library for building distributed apps with Redis (github.com) 31 points by thomas-st 10y ago ↗ HN
[–] ameyamk 10y ago ↗ This is good - but isn't this problem is now solved with redis cluster? [–] pjscott 10y ago ↗ Redis cluster is clustering for Redis; this is clustering for a certain class of other things, built on top of Redis.
[–] pjscott 10y ago ↗ Redis cluster is clustering for Redis; this is clustering for a certain class of other things, built on top of Redis.
[–] jwatte 10y ago ↗ The hash ring is so dumb. I don't understand why it's still a thing (or was a thing to begin with.)Just hash and mask some number of bits, and map each index you get to some node. Faster, simpler, even enough distribution.If your number of nodes is not a power of two, map multiple buckets to each node.When adding nodes, and the # buckets is too low, split each bucket in two by adding one bit to the mask.Consistent table construction means you only need the set of nodes to construct the take, keeping lookups entirely in core.Maybe I should blog this, but it feels a bit like "Hey you cab implement modulo power of two with a bit mask! Isn't that so crazy?" [–] Rhapso 10y ago ↗ You have invented kademila!As long as the network is "fully connected" by knowing all other members, topology will not really matter.If you use a Chord or pastry style skip list, you again have the exact same metrics as kademlia.All of these redis structures are really just re-branded DHTs. [–] thomas-st 10y ago ↗ What's "dumb" about the hash ring, and how exactly is your solution faster/simpler?One nice thing about the hash ring is that it nicely rebalances when you add or remove nodes. There is no need split any buckets.
[–] Rhapso 10y ago ↗ You have invented kademila!As long as the network is "fully connected" by knowing all other members, topology will not really matter.If you use a Chord or pastry style skip list, you again have the exact same metrics as kademlia.All of these redis structures are really just re-branded DHTs.
[–] thomas-st 10y ago ↗ What's "dumb" about the hash ring, and how exactly is your solution faster/simpler?One nice thing about the hash ring is that it nicely rebalances when you add or remove nodes. There is no need split any buckets.
6 comments
[ 4.4 ms ] story [ 19.6 ms ] threadJust hash and mask some number of bits, and map each index you get to some node. Faster, simpler, even enough distribution.
If your number of nodes is not a power of two, map multiple buckets to each node.
When adding nodes, and the # buckets is too low, split each bucket in two by adding one bit to the mask.
Consistent table construction means you only need the set of nodes to construct the take, keeping lookups entirely in core.
Maybe I should blog this, but it feels a bit like "Hey you cab implement modulo power of two with a bit mask! Isn't that so crazy?"
As long as the network is "fully connected" by knowing all other members, topology will not really matter.
If you use a Chord or pastry style skip list, you again have the exact same metrics as kademlia.
All of these redis structures are really just re-branded DHTs.
One nice thing about the hash ring is that it nicely rebalances when you add or remove nodes. There is no need split any buckets.