HN Explorer

Ask HN: Codebases with great, easy to read code?

234 points by impjohn ↗ HN
A colleague told me the best way to level up coding skills is to read excellent code.

Do you have favorite repos that highlight this?

I have an irrational fear of unknown codebases since it feels most of the code is either boilerplate or tied to some framework.

Do you have tips and tricks you use to read codebases?

142 comments

[ 4.0 ms ] story [ 182 ms ] thread
aitoehigie
This is a very interesting question.

Are you interested in any particular languages?

For Python, take a look at: https://github.com/psf/requests

impjohn
I initially had Python in the title but I removed it to give way to a broader discussion. Definitely checking this one out
johntdaly
To be honest, I don’t know any code bases I would call “great” or “easy to read” but I can tell you what I do when I need to work in codebases I don’t know.

I’ve got two main strategies:

1) I look at the part of the app I want to modify when I use the app and search for that part in the code. Once I’ve found that code I roughly try to find out how that code works by adding exploratory code (you can also use a debugger). Once I “think” I know what is going on I try to modify the code. This is where you usually find some exceptions or misunderstandings on you part if you haven’t touched the code before. If you are lucky and work in a team somebody can tell you in a code review that you didn’t understand. If you are alone you will have to see things blow up, debug and fix the problem.

2) You can try to figure out from the main entry point how the app works. This works better for some apps than for others. If you have an event based app this is most likely just a supplement to method 1, if you have a cli app or some type of data munching app this can replace method 1.

3) You can try looking at early versions of a code base in GIT to get an understanding of its architecture before the app became “more complex”.

You will always be a bit overwhelmed by any code base and many code bases are just to large for a single person so get comfortable working on “parts” of an app first rather than working on or understanding “the whole thing”. Also, code reading is not like reading books, code is way way denser than any book you can read (and that includes Heidegger) so you will not just “read” it, you will need to work with it. Zed Shaw’s “Learn X the Hard Way” series relies on you working with the code to understand it. The same holds true for code you “read”, you will at least need to try to “run” the code in your mind if you can’t run it for real.

You might also want to get over your thing about frameworks. QT, GTK, Ruby on Rails, React, ncurses, frameworks and libs are in just about any app and many apps that get larger might extract significant parts of their functionality into libs or frameworks. A lot of boilerplate is usually a good indication that an app could benefit from a framework. I never understood the “I want to be free from the constraints of frameworks” people. Their code bases usually have the start of multiple architectures and a lot of boiler plate code. I think they always search for some “perfect” solution and just can’t find it. The truth is, libs and frameworks are great, they give you an easy in on a new app and they give you documentation that probably wouldn’t exist on fully home grown code. In other words, they mace “reading” code easier.

eatonphil
I've been thinking a lot recently how to get devs to read more code and it's a very interesting reason you give that you don't want to wade through boilerplate. I never thought of that before.

I don't know boilerplate-heavy systems like Rails or Django too well. But I just wouldn't suggest starting with reading web app code (though maybe I've ignored reading too much web app code over time).

The easiest code to start thinking about is libraries and things you use today already like the nginx code base or the CPython code base or your logging library or your web server library code.

In these cases maybe you download the repo, build it, see how you could make a small tweak and run it. And soon you're looking through its code to understand how it works.

Another maybe easier technique to start reading more is when you are programming and have an error in a 3rd party library, use grep to find that error in 3rd library code and just start poking around when you do. Maybe add some print statements to it so you can see more of what goes wrong. Try to solve the problem just looking at the code and modifying it instead of using google.

If you ever get into it I'd love to hear from you. Email is on my site and Discord is in my HN profile.

Beltiras
Django and DRF are really well written systems with good documentation.
fernandotakai
absolutely +1 for django's source. and after they shipped their black integration, everything is properly formatted, which makes reading the source even easier.
bityard
I must be in the minority, I have tried to "learn Django" on more than one occasion and gave up every time after struggling with how hilariously over-engineered it is.

My best guess is that Django must be great for writing big important relational-database-backed apps, or rolling your own CMS, or something else people get a paid a lot of money to do. But personally, for my small projects I get more mileage out of starting with a micro-framework and just choosing and bolting on the bits I need.

Beltiras
That's exactly the case. My career has been mostly making CMS systems using Django. The neat thing is that you have this front-end batteries-included framework with a gargantuan ecosystem of system tools bolted on it's back end (Python). Django is not a web page tool. It's a tool to build large-scale web systems.
thijsvandien
I'm totally with you. Whenever I tried to build anything with Django, I soon went back to Tornado for its glorious simplicity. Being able to quickly see how something is implemented is worth much more to me than any magic I mostly end up fighting. Django just has too much going on with everything being somehow connected to everything else (which is sort of the point). I might agree it's well-engineered, but an example of easy to read code...?
tbrock
Redis. Read the redis source code if you want to see nice C.

The reason it always impresses me is that C can look like gobledygook, but yet this codebase is clean and understandable.

thorin
I was impressed with the Redis codebase too. I think it benefits from being relatively new in C terms so it doesn't have too much baggage (2009, is really new in terms of C projects!). It must also take a lot of discipline on behalf of the maintainer.

I seem to remember Postgres and Sqlite were relatively accessible to a low intermediate C programmer. When I've had to look at Android code (more C++ admittedly) I've started to get lost very quickly.

dkarl
I second Postgres. Not only is the source code a pleasure to read, there's also an unusual amount of well-presented material about its internals available online.
atonse
this probably correlates to why there is such a rich ecosystem of pg extensions and forks.
spacemanmatt
while true, the architecture rather than code hygiene deserves credit for facilitating the ecosystem of extensions and forks. in my husky opinion.
datavirtue
Also explains how Pivotal was able to refactor it into a MPP (Greenplum).
jjice
Postgres's Yacc definition helped me a ton when I was using Yacc. The documentation out there for Yacc/bison isn't great, but Postgres served as a decent set of examples.
bear8642
The Plan 9 operating system is good C codebase to explore too
zvr
If you're looking for code in C, the implementation of Tcl is a wonderful code base. You can even focus on specific parts instead of the complete scripting language: how to create a hash table, for example.
efrecon
I came here to say this. Redis and SQLite, mentioned elsewhere, have their roots in Tcl, so there are some connections.
erdeibit
Agree and not only Redis, the way Salvatore Sanfilippo (Redis creator) program is very readable and instructive. A glance into its repos worth the time.
numtel
Postgres
mattashii
Most sections of the codebase that are actively developed are very readable, but I still got quite lost in the core parts of xact/multixact recently. I feel that is more of an exception, though.
gorjusborg
I have found using github's language search to be helpful for this sort of thing.

If you are using ruby, for instance, just search for https://github.com/search?q=language%3Aruby and look for popular codebases. You can decide which are beautiful for yourself.

iamricks
How do you guys approach the "start" of reading a code base, i never know where to start looking, specifically if its a language i am not too familiar with i have no idea where to start and sometimes i have no idea where the program execution starts
miguendes
Mitchell Hashimoto has published a blog post describing how he approaches complex codebases. That might give you an idea where to start.

https://mitchellh.com/writing/contributing-to-complex-projec...

yodsanklai
Good advice! I especially like the first advice

> The first step to understanding the internals of any project is to become a user of the project.

It's normally easier to figure out complex behaviour from the spec/doc/interaction than from the code.

labrador
Great guide! I would add fixing bugs. I often learn most about a code base by fixing my bugs. A good debugger can be a blessing. Profiling is part of debugging to me. Questions can come up about why something is taking a long time that lead to more debugging and thinking about what is going on.
trynewideas
The best runs I've had working on others' codebases is to jump into documenting it. Many projects love having someone read, ask questions about, and document code, even (or especially) from a naive standpoint since that's who'll benefit the most from it, and in the process you learn how the code's structured, track where references lead to, and more often than not kick over some bugs worth fixing in the process.
yakubin
I've watched an interview on MSDN with one of the developers of .NET (I think she was responsible for the GC), who also used to work on Windows, making those famous workarounds making games work on newer Windows releases, even when they relied on old kernel bugs. I think she said that the best way to get familiar with a complex new codebase is to step through it with a debugger, going through several scenarios. I think it's a great idea. I only wish I had a working visual debugger in my day to day work.

EDIT: Found the interview: <https://docs.microsoft.com/en-us/shows/Careers-Behind-the-Co...>

Jach
Besides the good methods others have posted, and a really nice method (if you have it) of having someone else familiar with the code give you a tour, you can also do pretty well just by brute forcing it.

Get a list of all the files, sorted however (`find -name *.foo` works) and start going through them top to bottom, or bottom to top if that's a more clear convention of the language. Maybe shuffle order a bit if you discover unit tests (nearby or asking a tool to cross-reference a call) to read the code and the test around the same time, but resist the urge to jump around too much or too deeply. Jot down short notes about what seems to be the main purpose(s) of the file, and move on. Keep going, keep track of what you've seen, your first goal is to do a complete survey of all the files and not get too distracted by fully understanding new syntax (Java annotations and Python decorators can both be understood as high level declarative tags even though under the hood they're quite different) or endless note revisions from new insights as you progress and start seeing connections or just finally understanding terminology ("wtf is a 'hero'?").

You'd be surprised how fast you can do a single (high level, shallow, skimming in places) pass even for larger code bases, by the end of it you'll also have found the/an entry point, and are in a better place for followup study or producing materials that can help the next person (like an architecture diagram that lists the files involved in each element, at least at that moment, or just some important cross references you've noted that a tool isn't necessarily going to make clear). And for easy code, a single pass may be all you ever need, even if you read it in a strange order. A completed puzzle is perfectly clear regardless of the order you put the pieces down.

ramboldio
GRBL the CNC firware for Arduninos:

https://github.com/grbl/grbl/

It feels like it has more comments than code. The comments are written in a very nice, understandable language that even activley teaches about concepts that are only adjacent to the code at hand.

E.g. https://github.com/grbl/grbl/blob/master/grbl/stepper.c#L142 or https://github.com/grbl/grbl/blob/master/grbl/stepper.c#L233

jovial_cavalier
It's funny this got mentioned, because I recently got a 3D printer that runs Marlin, which embeds GRBL. So, I decided I would take a look at it. I thought a lot of places were really garbled. Especially motion_control.c, which has a ton of #ifdef logic
bcrl
That's over-commented code written by a junior developer from my quick look. The first random thing I looked at in grbl/eeprom.c:

... char old_value; // Old EEPROM value. char diff_mask; // Difference mask, i.e. old value XOR new value.

cli(); // Ensure atomic operation for the write operation. ...

You can remove the need for the first comment by calling the variable old_eeprom_value. Boom, simple and obvious. Commenting cli() is similarly ridiculous: call the function disable_interrupts() and it's completely obvious what it's doing. Later on:

sei(); // Restore interrupt flag state.

This is incorrect. It's enabling interrupts, not restoring them. If the intent was actually to restore the interrupt disable flag to its original state then this function is buggy and will unintentionally enable them. It would be far better to document the expected sematics in the documentation for the function above, but instead of documenting the expected semantics of the eeprom_put_char() function, you have to read the code to figure out what the semantics are. What would be better is to have a comment in the function description saying "this function can only be called with interrupts enabled" or "this function is atomic and can be safely called from an interrupt handler or with interrupts enabled". Then it's obvious when reading the code which semantics are guaranteed / expected.

So, sure, overly commented code makes it easy to figure things out, but this is a sign of a junior developer that is focused too much on the code and not enough on the overall system. This isn't something I'd like to see a developer pointed at that is looking to learn good habits. Good habits are telling other developer what they can expect from a function. Bad habits are making them read the code to figure that out.

simonw
Something I find really helpful is to start with a question that I want to answer.

Often this will be along the lines of "How does it do X?" - where X is something I either didn't know was possible or that I suspect to be really difficult.

Then I can dive in to the codebase (usually starting with GitHub code search) and try to figure out how they do it.

This helps me skip straight past the boilerplate and means I often get to a satisfying conclusion - where I've learned something new - in a very small amount of time.

And along the way I pick up knowledge about how their code is organized and often a few other tricks too.

simonw
One recent example: I wanted to know if the SQLite package in Python took any steps to avoid calling "interrupt" on a closed connection, which the SQLite C documentation warns against.

A couple of searches against https://github.com/python/cpython lead me to this code here: https://github.com/python/cpython/blob/4674fd4e938eb4a29ccd5...

gerhardhaering
It's nice that code that I wrote more than a decade ago is mentioned here.
edelans
And it's even nicer that you get the chance to see it !
simonw
Thank you very much for building this, I benefit from it every day!

Since you're here... there was actually a question raised on the SQLite forum about that code and whether it is genuinely safe against a specific race condition... and I don't have nearly enough Python C knowledge to know the answer!

Does this look like it could be a problem to you? https://sqlite.org/forum/forumpost/f37ae374cc

jorangreef
Are we allowed to share repos we've written? :)

If so, then here's distributed consensus in Zig:

https://github.com/coilhq/tigerbeetle/blob/main/src/vsr/repl...

Something that differentiates this from many consensus implementations is that there's no boilerplate networking/multithreading code leaking through, it's all message passing, so that it can be deterministically fuzz tested.

I learned so much, and had so much fun writing this, that I also hope it's an enjoyable read—or please let me know what can be improved!

Mawr
Very clean, tasteful formatting. Easy to read. There's even correct splitting of long argument lists over multiple lines: [1], [2]. You don't see that very often, most programmers have awful taste about this.

I have some qualms with these one-line early returns though:

  if (commit <= self.commit_min) return;
Control flow statements should always be on their own lines, then it's easy to find all of them by visually scanning top-down, without needing to look all the way down each line.

[1]: https://github.com/coilhq/tigerbeetle/blob/main/src/vsr/repl... [2]: https://github.com/coilhq/tigerbeetle/blob/main/src/vsr/repl...

mbrock
Zig's code formatter puts list items on separate lines if the last item has a trailing comma; otherwise, it puts them all on the same line. So if you use the code formatter, you'll either have just one line, or one line per item. It's pretty nice.
aidos
Look through the stack you’re familiar with. For me that means nginx, uwsgi, flask, sqlalchemy, alembic - but I’ll look at anything I have a question about.

My trick is to dig in when something doesn’t work the way I expect. Or someone says “I don’t think there’s a way to do X with blah”. My immediate reaction is to clone the code and take a look. I have a “tools” folder on my local machine that contains many of the tools / libraries is use.

Orientation is easier than you expect. The easiest scenarios are around “why did I get that error” situations. Grep for the error and away you go. But having a question to answer will definitely give you a direction to investigate.

gmassman
I agree about having an error to investigate or a question to answer. Some of the python libraries with great code I'd recommend reading are Flask and Werkzueg. Both have very clean interfaces and excellent documentation. Sqlalchemy is somewhere in the middle. As an ORM it has a high minimum level of complexity. But the codebase is still reasonably well organized. Looking for an answer to something like "how does it emit a JOIN" may be a lot harder to answer than you expect though.

For packages to avoid, stay away from Celery. It's just... icky.

aidos
Yup, sqla is super meta and, thus, complicated (just due to the problem space). Better is alembic / alembicutils where you can dig into the autogenerate system they’re layering over sqla.

Weirdly enough, I enjoy digging through C and Java libs more, mostly because they’re more unfamiliar to me. I’d spend more time in Postgres / fontforge / mupdf / pdfbox / nginx / uwsgi on the whole.

mekoka
I think the recommendation to study Flask's code base is an echo from years ago, when it was considered a micro/toy framework. I used to recommend it too. As a more seasoned programmer, I don't anymore. Having used and studied the framework, I've seen some of the limitations that directly stem from some of its design choices. Today, I think it tried to be needlessly clever sometimes. Some of those choices, that seemed fun and clever back in the days, did not age well. As time went and people started pushing the framework, some artefacts required hackish workarounds. And those kept cumulating. To such an extent that the codebase now has a bunch of flags that control it's contingent behavior; ifs and buts that make the whole thing harder to reason about. You'll still learn lots of clever Python tricks, but I wouldn't put it in the "clean and clear" category.
srvmshr
Depending on your interest, I could vouch for OpenBSD having a very clean readable codebase. Often it has some of the best practices coded in with useful commentary.
ramesh31
Doom 3 is a perennial favorite for "most beautiful C++ codebase" lists [0]

[0] https://github.com/id-Software/DOOM-3-BFG

dafelst
I was going to comment the same thing, all the Carmack era id software open source code (Quake, Doom) is very nicely structured and quite easy to grok.
anonymoushn
The zig stdlib has been good reading so far. You also basically have to read it if you want to use it.
amichal
https://github.com/seattlerb/minitest really removed the FUD for me when i started learning Ruby and Rails. Its full of metaprogramming and fancy tricks but is also quite small, practical and informal in its style.

e.g. "assert_equal" is really just "expected == actual" at it's core but it uses both both a block param (a kind of closure) for composing a default message and calls "diff" which is a dumb wrapper around the system "diff" utility (horrors!). There is even some evolved nastiness in there for an API change that uses the existing assert/refute logic to raise an informative message. this is handled with a simple if and not some sort of complex hard-to-follow factory pattern or dependency injection misuse.

https://github.com/seattlerb/minitest/blob/master/lib/minite...

cfcosta
That's an oldie, but the experience was the same for me. I was reading Metaprogramming Ruby at the time, and going over the implementation made everything much clearer to understand.
mongrelion
I can't believe it. It was the same for me back in 2009 when I was learning Ruby. Thanks for this.