HN Explorer

Ask HN: What are some good projects/resources to learn distributed systems?

54 points by gyani95 ↗ HN
I'm a kind of person who likes to hack on projects to learn things. If I am learning a new language, I'll think of a problem that I want to solve. After reading some syntax I'll start hacking on a solution.

I have decided to learn distributed systems.

I think this weekend would be a good start to hack on things. I can continue later in the week I understand I won't pickup distributed systems over a weekend.

What resources do you recommend for one trying to learn distributed systems on their own?

14 comments

[ 4.9 ms ] story [ 41.7 ms ] thread
jxub
I'd recommend the book Designing Data Intensive Applications, by Martin Kleppmann and as for online resources, check out http://book.mixu.net/distsys/single-page.html for a throughout overview.
gyani95
Thanks for the suggestion. I've added it to my list. Do you recommend some good first apps for learning distributed systems?
iampims
This Github repo is a pretty good start to get familiar with distributed systems:

https://github.com/donnemartin/system-design-primer

gyani95
I've been through this. It's a good starting point and can be fairly useful for interviews. It gives a good high level overview of systems, enough for you to talk about them.

Maybe I can go through this again and dive deeper into topics that interest me.

convolvatron
do you want to learn how to design distributed systems from first principles or build systems from pre-existing distributed components?

unfortunately these are both called 'distributed systems'

#1 is alot more interesting. #2 is what people are hiring for

gyani95
I'd like to say both! I guess I can get started with the second one and then move onto the first one?

Maybe after playing with a few components I can look into their internals.

dsnuh
For #2 - Maybe try setting up your own Kubernetes cluster on a cloud service provider, without using shortcuts? Perhaps etcd and the Raft protocol would be interesting and helpful in some cases, but leans more towards #1.

For #1 - CAP Theorem, read through aphyr's blog, google concepts/details/jargon you find in white papers on different of the various distributed consensus protocols, etc.

daoran
Beside Martin Kleppmann's wonderful book, I'd also recommend MIT 6.824 distributed system course, you can find lots of resources on the course website.
lambda_func
Some stuff you can build to learn more about distributed systems (you can start simple, do research, and then iterate to build a more comprehensive solution): build a distributed file system, a distributed lock server, a consensus algorithm (paxos, raft, etc.), two/three phase commit protocol, distributed hash table, etc.
simon_acca
Aphyr, of Jepsen fame, published an outline for a distributed systems class that makes for a good reference on what is relevant for most modern distributed systems: https://github.com/aphyr/distsys-class
gyani95
Thanks for this! This sounds lovely and is 12-16 hours long. Fits my schedule :D