HN Explorer

Ask HN: Which are the best Go repositories to read to learn the language?

199 points by Kreotiko ↗ HN
I am looking for examples of open source projects showcasing best in class use of Golang. It doesn’t need to be the famous products we all know about.

I am looking at examples of real world professional usage for somebody already familiar with the language constructs and syntax.

50 comments

[ 73.6 ms ] story [ 2573 ms ] thread
jszymborski
It's no substitute for reading the code for a useful bit of software, but I always like to head over to https://learnxinyminutes.com/ to check out the short code examples/overviews to get a feel for a language.
BlueGh0st
Their COBOL in 30 minutes was a nice before bed read that put me well informed enough to have a discussion with my mom about the details of her career and looking over notes from her work with the language.

I've been intending to look up Golang on the site but I forgot until now. Thanks!

nemo1618
The best Go source code is the stdlib. When you read stdlib code, you can be confident that nothing is there by accident. Every decision was made for a reason. When you're learning, that's invaluable, because it means you can "dig" anywhere and be rewarded for it, whereas "digging" into most codebases will often be a waste of time.

Also, I would advise actively avoiding the big names you have heard of. A lot of products are successful despite having garbage code. And even more of them are successful despite having only decent code, full of stuff not worth emulating.

Try to identify people who have a lot of experience and a strong command of the language, then look at their most recent projects.

thinkingkong
The stdlib is the best for clarity. But there are some excellent examples of how to structure projects depending on what you're trying to build. Mat Ryer gave an excellent talk on writing HTTP Services. If you want a good example on how to write distributed systems then Hashicorps' projects are more or less gorgeous.

https://medium.com/@matryer/how-i-write-go-http-services-aft...

https://github.com/hashicorp

matwood
To add to your point, Nic Jackson from hashicorp has a bunch of how-to videos and presentations that I've found good also.

https://www.youtube.com/channel/UC2V1SxXFUa5YxVJvTsrCgyg

dastx
Great resource! Thanks.
nemo1618
> Hashicorps' projects are more or less gorgeous

Is there a particular piece of code (as in, a single package/file/function) that jumps out to you as "gorgeous?" When I look at a random Hashicorp repo, it strikes me an impenetrable labyrinth.

aprdm
Try to add a new feature or see how some feature works, with consul I did some greps over some features I used and followed the code.. was able to put a merge request upstream changing a behaviour I didn't like (configurable) within 4h (that didn't get accepted and that's OK)
tonymet
It’s also one of the more readable stdlibs. I use it for idioms and snippets .
matwood
> The best Go source code is the stdlib.

As a golang beginner, the go stdlib is one the most readable stdlibs I've read through. The combination of gofmt, manageable language features, and idiomatic ways to do things make most go code fairly easy to read.

I know go doesn't have a million language features, but readability after the fact is one of the benefits of this approach.

sagichmal
The stdlib is kind of a mess. One kitchen, many cooks, and everyone was learning how to put a five-course meal together as they went. There are some great bits -- package io and ioutil, for example -- but a lot of it is full of what we now understand are bad ideas, and the compatibility promise prevents anyone from doing anything about it.

There is much better code to read. I second the recommendation of HashiCorp's stuff. Stay away from Docker and especially Kubernetes.

geoah
Couldn't agree more. Cheney's response to why that is was "do as we say, not as we do".
wbl
Do you have source / more details? There certainly are a few warts but overall it gets the job done.
earthboundkid
FYI, ioutil is deprecated in Go 1.16. :-) They copied everything out of it and into io and os. It’s much cleaner actually once you get used to it.
maerF0x0
Another reason it is a bit of a mess is because of the no breaking changes promise. They have to keep ugly or deprecated interfaces unchanged, even into Go 2, iirc.
phoinix
I second that. Stdlib is great to get a feel how the language works. Some small programs are always the best, if they are written by a good programmer. A cli game i study right now but in Rust. A small program with a lot of logic, but without all the graphics distractions.

https://github.com/VladimirMarkelov/solkit

ValentineC
I consider rclone one of my most used tools for managing data, and personally think it's reasonably well-structured: https://github.com/rclone/rclone

I agree with the other commenters that stdlib is excellent for delving deep into the language.

tyingq
I think LXD would be a good repo to look at. You would see examples of REST daemons, clustering, database access, auth, containers, etc. https://github.com/lxc/lxd
37ef_ced3
Read the Go standard library, especially the parts written by Russ Cox. The parts written by Griesemer are less pleasant, no offense to him

But some of Go's language decisions only become clear once you've written a lot of Go

For example, the "var name = expr" declaration form seems unnecessary in light of the short ":=" form but it is in the language to allow indented declaration blocks like this (where some variables get an initial value and some get just a type):

  var (
      text []byte
      last int
      more = true
  )
  ...
I've been using Go for 3 years and the more Go I write, the more I appreciate it. Enjoy
vegancap
I second Russ Cox's contributions to the stdlib. I recently went to check which version of Go the embedded API was added, and noticed a PR by Russ, I glanced through it and saw some really impressive and thoughtful code. I've made a point since to go back and look through more!
JohnCClarke
The easiest way to learn golang is to sign up on HackerRank and try out their exercises. You can do it in-browser, so no need to even install the toolchain. You'll learn faster by doing and reading in parallel.

To see examples of good source code just click thru the function names when searching the documentation. E.g. https://golang.org/pkg/net/http/#NewRequest They link directly to the source code, and the standard library is extremely well commented.

marvinblum
You can check out the Pirsch Analytics core [0]. I'm the maintainer and can answer all your questions :) It probably contains most features you'll use when programming in Go. Take a look at the tracker.go for a concurrent collector. It also has a lot of parsing, a data model, database, http, public API, ... and UI is not required while learning Go I suppose.

[0] https://github.com/pirsch-analytics/pirsch

deepanchor
I learned a lot from reading the Perkeep [1] codebase (formerly Calimstore). One of its core developers is Brad Fitzpatrick (former Golang team member) so you can be fairly confident it’s idiomatic Go.

[1] https://github.com/perkeep/perkeep

stanislavb
Hey mate. I built, recently, a service that tracks all repositories mentioned on HackerNews and Reddit. Based on that data you can see which are the most popular or talked about repos per language. I guess that can help you. e.g. https://www.libhunt.com/lang/go
pa7ch
I really enjoyed reading the upspin codebase. I thought it was really a clean design and clean code. Some of the early go team members were working on it including andrew gerrand and rob pike. I think its a bit of an experiment on how one might design a content sharing system if you were to start from the ground up and not on the current web standards. I particularly like how they work authentication and authorization into the model. I had a few experimental upspin services I wanted to write but never got around to.

Another great read for learning go that covers the more traditional challenges of standing up a basic web service with user accounts, testing, metrics, etc. is https://github.com/benbjohnson/wtf and the accompanying blog posts that cover how to structure such an application and why.

bilinguliar
I also suggest reading: stdlib, projects by Rob Pike (Upspin), Andrew Gerrand, Brad Fitzpatrick, Dmitry Vyukov, Jaana Dogan, Jonathan Amsterdam.

Please avoid: Kubernetes, AWS code.

As a rule of thumb - less imports is better.

m1
Just out of interest, why avoid Kubernetes?
bilinguliar
Do not get me wrong, k8s is a revolutionary piece of software, but it is not the best example of a clean Go project.

There are a few disturbing things:

1) Usage of `utils` package. Points against it were made in:

https://dave.cheney.net/2019/01/08/avoid-package-names-like-...

https://github.com/golang/go/wiki/CodeReviewComments#package...

2) Check out the number of dependencies. This looks almost like NPM hell.

3) Lack of standardization and consistency. This project is so big, and there are so many contributors with their coding styles that the project looks untidy.

4) A lot of import renaming is a symptom of package name clashing or simply poor package naming. You usually do not need to rename import, and this is simply not required if your package structure was carefully thought about.

Again, k8s is great. K8s solves a lot of problems. I love it, but it is not a great example of Go code.

theshrike79
Kubernetes started with a really early version of Go and had to create from scratch a bunch of stuff that's in the stdlib nowadays.

Basically they've reinvented the wheel a bunch of times and now it's too late to import stdlib.Wheel =)

nvarsj
Actually, k8s was written in Java originally. Then basically verbatim copied to golang. That’s why you have this ObjectMeta stuff - it was trying to implement an OO system in golang. Lots of weirdness can be traced back to the Java origin.
smarterclayton
That’s not true, because I’m the original author of ObjectMeta and I did it in golang. :)

Half of the point of Kube is that every object looks similar enough that you can deal with them generically. ObjectMeta is what enforces that, so yes it’s OO, if there was only one base class possible and no inheritance. The ObjectMeta as an interface construct exists for performance reasons in Kube vs the reflection we used early on (which was mostly due to people learning Go and dealing with limitations of the language as we rapidly prototyped new ways of building out our object patterns).

A prototype of kube was done at google in Java, none of that survived translation to the open source project (edit: or was rewritten prior to v1 as we matured the subsystems that replaced the basic concepts in the day 1 project, like clients writing directly to etcd). Many of the early authors had lots of Java experience, but there was never any attempt to bring Javaisms into Kube or any Java to be converted, just conventions.

EDIT: actually, metadata/spec/status is more a repudiation of Rails and the active record style “hundreds of properties with no organization” JSON apis. We wanted to clearly separate metadata (common), spec (desired intent), and status (actual state). That led naturally in go to an embedded struct for ObjectMeta and then the interface for getting access to the generic properties on any object.

droctothorpe
This. This is why I read hacker news. As someone who interacts with your code on a daily basis and whose livelihood was catapulted due to said familiarity, thank you!
earthboundkid
I didn’t see anyone else mention AWS, so let me add: AWS is a tire fire of bad API design, in both v1 and v2. Just awful, awful stuff. Read it and weep.
elseweather
What's wrong with it? I'd totally believe it's a mess, but I'm curious what kind of mess. Specifically I'm wondering if it's "bad Go code" or the underlying AWS APIs (or both!)
mbyio
A lot of people are saying to read the stdlib, but the stdlib has a lot of objectively bad code in it. Some of it is necessary due to the nature of a standard library - you have extra concerns about dependencies to ensure people don't have to include the entire stdlib in every binary. Some of it is stuff they hacked together when they were still designing the language, and that initial code still works fine so no need to update it.

I think the source code for pkg.go.dev is actually a treasure trove of good ideas and design patterns, especially if you are trying to make a small to mdedium sized web app (which, lets be real, is almost all web apps anyone will ever make). https://github.com/golang/pkgsite

achiang
I haven't seen anyone recommend net/http yet, but that is the library that was recommended to me to read to get a sense for modern, idiomatic go.
[deleted]
(comment deleted)
aliyfarah
I recommend the Mattermost server:

https://github.com/mattermost/mattermost-server

And I disagree with those that say stdlib is the best way of looking at the best Go code for two reasons:

1. A lot of times stdlib code is restricted to use only backward compatible code with old API contracts, some parts are neat but others are unwieldy. It's a hit or miss.

2. Library code is different to application code. You can't get as much variety in style, abstractions, design patterns in stdlib as you can in a real world application.

I reckon it's important to read to get a whiff of succinct Go code, very good at learning protocols (like OAuth, Http etc) if you're interested, but won't be hugely helpful in building a CRUD app.