"Understanding _using_ singleflight in Go" would be a better title. This generated article doesn't give the reader a real understanding of the implementation and its various tradeoffs that you might care about depending on your workload (e.g. should the first execution to reach a key spawn a goroutine or is that allocation too much)
That being said, singleflight is a fantastic library and pattern that helps so much with p95 latency. It's a little noisy code-wise, but if you use it in the right places the gains are huge.
Also, totally agree with the below comment that recommends janos/singleflight -- start there, but most of the critical projects at my company, AuthZed, have reimplementations with tailored semantics.
> This pattern is particularly useful in scenarios where caching isn't suitable or when the results are expected to change frequently
Any suitable examples? In the linked posts it gives querying weather service as an example but still uses a cache. Even with a normal concurrent request to a single function, a caching layer can be added before it makes any external request. Or am I misunderstanding this use case?
I think the idea is to avoid problems with functions or APIs that have side effects. You want to make sure you perform the task only once. Or you want to make sure you’re not burning money on some commercial API through redundant calls. It‘s not necessarily about performance, hence why the weather service example also uses an additional cache specifically for performance.
The problem with singleflight is that when a goroutine that grabs it first happens to be a slow request, it slows down entire group. What's worse is if it returns error, rest of group share the fate. That sometimes shows in bursts of cancellations on one of goroutines timing out.
Better would be if it was semaphoreflight e.g. allow up to N concurrent requests to proceed to amortize this.
Another issue is there's per group lock that can generate meaningful contention just to check the result.
7 comments
[ 3.1 ms ] story [ 47.4 ms ] threadThat being said, singleflight is a fantastic library and pattern that helps so much with p95 latency. It's a little noisy code-wise, but if you use it in the right places the gains are huge.
Also, totally agree with the below comment that recommends janos/singleflight -- start there, but most of the critical projects at my company, AuthZed, have reimplementations with tailored semantics.
https://github.com/janos/singleflight
I then open sourced it in Jan 2013 in what was then named Camlistore (now Perkeep) in https://github.com/perkeep/perkeep/commit/6f9f0bdda9c9c1f147... d
And later I put it in https://pkg.go.dev/github.com/golang/groupcache/singleflight (groupcache was written for dl.google.com)
And a private copy in Go's net package in Jun 2013: https://github.com/golang/go/commit/61d3b2db6292581fc07a3767...
It later moved to golang.org/x/net, and later to the Go standard library (well, internal: https://pkg.go.dev/internal/singleflight)
We now even have a copy with generics in Tailscale's tree at https://pkg.go.dev/tailscale.com/util/singleflight
So many variants of that code :)
Any suitable examples? In the linked posts it gives querying weather service as an example but still uses a cache. Even with a normal concurrent request to a single function, a caching layer can be added before it makes any external request. Or am I misunderstanding this use case?
Better would be if it was semaphoreflight e.g. allow up to N concurrent requests to proceed to amortize this.
Another issue is there's per group lock that can generate meaningful contention just to check the result.