So it's really the issue due a partial variable shadowing during assignment, not errgroup-specific. Got bitten by it multiple times unfortunately, but I don't think there's an easy lint or vet check that could prevent such errors from happening. Having said that, overwriting a variable during := assignment is typically only useful for errors, so potentially you might want to have a lint check that complains when you've overwritten something else. There's already a check that the value is unused so when re-assigning the error value you're less likely to miss anything
I'd say it's fair to call this a footgun, though not a bug. The context is really only intended to apply to the goroutines. And Wait has to cancel the context to prevent a resource leak.
I suggest in general using function scoping to drive the lifetime of contexts, etc. This works also for defers and tracing spans in addition to the canonically shadowed `ctx` variable.
There is an old issue in the tracker proposing changes to alleviate this: https://github.com/golang/go/issues/34510. The original author (bcmills) of the errgroup package shared insight into the design choices/tradeoffs he made.
> Shadowing is another concept that made this issue less visible.
There is no shadowing here, Go can only shadow in different scopes, `ctx` is just rebound (overwritten). Zig does not prevent reusing variables that I know. However I believe zig parameters are implicitly `const`, which would have prevented thoughtlessly updating the local in-place. Same in Rust.
> If you've ever heard of linear types, and never saw their utility, that's actually exactly what they are good for: a variable gets 'consumed' by a function, and the type system prevents us from using it after that point. Conceptually, g.Wait(ctx) consumes ctx and there is no point using this ctx afterwards.
I don't believe linear types would prevent this issue per-se since `errgroup.WithContext` specifically creates a derived context.
Depending on the affordance they might make the issue clearer but they also might not: a linear errgroup.WithContext could consume its input leaving you only with the sub-context and thinking little more of it, so you'd have to think to create derived context on the caller side. Would you think of this issue when doing that?
TBH this seems more of a design issue in errgroup's API: why is the context returned from the errgroup instead of being an attribute? That would limit namespace pollution and would make the relationship (and possible effects) much clearer. But I don't know what drove the API to be designed as it is.
But in all honesty I'd say the main error in the snippet is that `checkHaveIBeenPawned` swallows all http client errors, without even logging them. That is the part which struck out to me, it only returns an error if it positively finds that the password was a hit on HIBP. So you'd have hit the exact same issue if the `NewRequestWithContext` was otherwise malformed e.g. typo in the method or URL. And of course that points out to an other issue, this time with type-erased errors and error information not necessarily being programmatically accessible or at least idiomatically checked: I would assume the goal was to avoid triggering an error if HIBP is unavailable (DNS lookup or connection error).
This seems like a terrible misunderstanding of the errgroup package? The entire point of the returned context is that it's cancelled if an error occurs, yet the author never uses it again until after he's done with the errgroup. The behaviour is also clearly documented in the only function exported by the package.
10 comments
[ 3.1 ms ] story [ 33.3 ms ] threadI suggest in general using function scoping to drive the lifetime of contexts, etc. This works also for defers and tracing spans in addition to the canonically shadowed `ctx` variable.
There is an old issue in the tracker proposing changes to alleviate this: https://github.com/golang/go/issues/34510. The original author (bcmills) of the errgroup package shared insight into the design choices/tradeoffs he made.
There is no shadowing here, Go can only shadow in different scopes, `ctx` is just rebound (overwritten). Zig does not prevent reusing variables that I know. However I believe zig parameters are implicitly `const`, which would have prevented thoughtlessly updating the local in-place. Same in Rust.
> If you've ever heard of linear types, and never saw their utility, that's actually exactly what they are good for: a variable gets 'consumed' by a function, and the type system prevents us from using it after that point. Conceptually, g.Wait(ctx) consumes ctx and there is no point using this ctx afterwards.
I don't believe linear types would prevent this issue per-se since `errgroup.WithContext` specifically creates a derived context.
Depending on the affordance they might make the issue clearer but they also might not: a linear errgroup.WithContext could consume its input leaving you only with the sub-context and thinking little more of it, so you'd have to think to create derived context on the caller side. Would you think of this issue when doing that?
TBH this seems more of a design issue in errgroup's API: why is the context returned from the errgroup instead of being an attribute? That would limit namespace pollution and would make the relationship (and possible effects) much clearer. But I don't know what drove the API to be designed as it is.
But in all honesty I'd say the main error in the snippet is that `checkHaveIBeenPawned` swallows all http client errors, without even logging them. That is the part which struck out to me, it only returns an error if it positively finds that the password was a hit on HIBP. So you'd have hit the exact same issue if the `NewRequestWithContext` was otherwise malformed e.g. typo in the method or URL. And of course that points out to an other issue, this time with type-erased errors and error information not necessarily being programmatically accessible or at least idiomatically checked: I would assume the goal was to avoid triggering an error if HIBP is unavailable (DNS lookup or connection error).
That would have avoided the problem.
Perhaps worth a submission to https://staticcheck.dev/ ?