Sadly ringpop is no more active. But in golimit we are using just clustering constructs of ringpop, which seems to be quite stable in latest version.
We are using distributed counter for rate calculation. Every node stores a local and global counter value. Rate limit is applied on local+global value.
The local value is synced periodically on configured interval or when local counter threshold exceeds a configured threshold value.
If the underlying project is unsupported, and you're only using a part of it, and the licenses permit, I'd suggest just pulling the relevant code into your own. That eliminates the underlying criticism of "depending on code that no one is maintaining" by showing that you will maintain it.
Plus, if you stick with this project and start nursing it through the ever-increasing demand for performance, you're pretty much guaranteed to do this eventually anyhow, so there's not much reason to put it off. :)
If I’m reading this right, is it possible to use the sidecar technique to install this along keep my existing load balancer architecture? For example, if every server has an haproxy frontend, and I install this sidecar on each server, how do I interact with it from haproxy?
Why does this implement an HTTP interface on the sidecar? Isn’t that a lot of unnecessary overhead for incrementing a counter? The connection will always be on a local interface, so there’s no need for HTTP. Seems like a better solution would be a simple Linux domain socket, which would be far less overhead for the eg haproxy <-> sidecar communication.
absolutely right, Unix sockets are quite faster than even loopback tcp connections. We have plan to introduce that in roadmap.
But to start with we wanted to cover general usecases first. Http interface also make it platform independent.
Infact loopback tcp connections are also giving decent performance as the requests are routed though different interface
If the deployment is using docker (i.e. haproxy in one container, the sidecar in another), “local” TCP can be a big bottleneck. I was using TCP for redis over a local docker network, and got a huge speedup after switching to Linux domain socket mounted in a shared volume.
Anyway, the project looks nice, I will be checking it out. But for production deployment the local HTTP would be a dealbreaker for me.
Skimming the bucket code, it appears that the rate-limiting information is stored in a map. It doesn't appear to implement an LRU or attempt to clean up stale entries.
If I feed this user-supplied keys such as IP address or cookies, doesn't this mean it will grow without bound?
In addition to the 15 bytes used by the string itself, which will be rounded up to a minimum of 16 bytes in RAM (and probably larger), the string carries additional words for pointers and data, whereas the int is just 4 bytes and done. Go can use it as a value type so there doesn't even have to be another couple of words wrapping it as an "object" or something. Plus you can potentially start optimizing for the int case with various clever things more easily, because it's so tight it gets down into the realm where bit bashing games can be used to potentially avoid cache line hits, etc. whereas the string representation isn't so amenable to that.
Golimit tries to reuse the allocated keys counter entry in case the counter has expired and its not collected by GC.
There is a very simple GC logic where GC is invoked periodically, and scans for all buckets one by one and cleans them up. The GC invocation frequency can be configured by setting "gcinterval" property in yaml.
Why is it impossible, maybe just not useful? Otherwise you can use CRDTs for connection counters and maybe exclude counters that are lagging behind too much.
The project looks cool. Just want to understand how counter syncing works here, have you implemented CRDTs? Since, everything is in RAM, if service goes down, all counters' data will be lost. I believe, in case of API gateways some kind of durability is needed, if we need to rate limit based on consumer keys.
Thanks, The counters are synced in CRDT way , every node has both local and global counters and local counter are synced asynchronously with global counter periodically or on certain events.
Usually the rates would be defined in a period of few minutes or seconds. So after the window is over the counter value will vanish automatically. so making counter durable seems unnecessary overhead as of now. But durability can be added in project if the need comes
24 comments
[ 5.1 ms ] story [ 39.2 ms ] threadHowever it seems based on https://github.com/uber/ringpop-go which is marked as no longer active...
What are you using to implement rate limiting ?
We are using distributed counter for rate calculation. Every node stores a local and global counter value. Rate limit is applied on local+global value. The local value is synced periodically on configured interval or when local counter threshold exceeds a configured threshold value.
Plus, if you stick with this project and start nursing it through the ever-increasing demand for performance, you're pretty much guaranteed to do this eventually anyhow, so there's not much reason to put it off. :)
Also, there was an HN thread last year from stripe with a good discussion of distributed rate limiting: https://news.ycombinator.com/item?id=13997029
Anyway, the project looks nice, I will be checking it out. But for production deployment the local HTTP would be a dealbreaker for me.
If I feed this user-supplied keys such as IP address or cookies, doesn't this mean it will grow without bound?
https://aphyr.com/posts/277-timelike-a-network-simulator
https://www.youtube.com/watch?v=kpvbOzHUakA