HN Explorer

Ask HN: “Write your own” or “Build your own” software projects

576 points by n_t ↗ HN
I am looking for writings/tutorials/videos which describe a specific technology or feature by implementing them, ideally in no more than few thousands lines of code (and not just 10-20 line code snippets). Idea is to teach about underlying technology by a hands-on project, which is not overwhelming like trying to implement full-feature game engine and yet captures the essence of technology. Some examples are -

* Build a simple database (https://cstack.github.io/db_tutorial/)

* Containers in 500 lines (https://blog.lizzie.io/linux-containers-in-500-loc.html)

* Malloc tutorial (https://danluu.com/malloc-tutorial/)

* Nativecoin - build your own crypto-currency (https://lhartikk.github.io/)

I'm sure there are great such projects/tutorials in domains like networking, filesystem, databases, compiler, web design, messaging, game design, fintech, etc. If you have come across such writings/projects, kindly share.

86 comments

[ 3.1 ms ] story [ 82.5 ms ] thread
brudgers
Building a Modern Computer from First Principles http://nand2tetris.org/
n_t
It certainly is a good course and I recommend it too. However, this is a full-fledged class requiring many weeks instead of few hours or days.
SilasX
I was on the fence about whether to suggest it, but I don't think you should dismiss it as being a weeks-long course.

Instead, think of it as 12 mostly-independent projects implementing a small part of a computer's functionality: the set of logical gates, the ALU, the CPU, the computer hardware, the assembler, assembly programming, stack op implementation, a compiler, and and OS. Any one of those will be only a few days.

(I'm in the middle of the Coursera course for it and have done 1-5 and 7-10.)

http://www.nand2tetris.org/course.php

It only has up to chapter 6 online, but there's an old version of the book available for free but I don't have the link handy.

otras
I worked through most of this tutorial on building a simple text editor in C and greatly enjoyed it. Highly recommend it!

https://viewsourcecode.org/snaptoken/kilo/

n_t
Good one!
_diyu
Not just a basic one either, it had syntax highlighting! I always thought that was a pretty involved and complex feature, but it's simplified and demystified here.
herbstein
It's still very early days, but Bitwise is interesting (https://github.com/pervognsen/bitwise)

There's also Handmade Hero (https://handmadehero.org/)

The Raytracing books by Peter Shirley are also very interesting, starting with "Raytracing in one weekend" (https://www.amazon.com/Ray-Tracing-Weekend-Minibooks-Book-eb...)

And lastly there's Crafting Interpreters (http://www.craftinginterpreters.com/)

abhishekjha
I am following bitwise very closely, particularly to get better at C. Any companion text/resource that I should follow along?
herbstein
The Oberon book by Wirth is something Per references quite often, and is also linked in the README. I think that is the only companion text you'll really find. Other than that it's reading up on the concepts in general from different sources.
jeffreyrogers
NAND to Tetris[0] has similar goals, but is probably significantly different in terms of implementation that it's only helpful to explain the concepts.

[0]: http://nand2tetris.org/

runevault
I'm not quite through part 1 yet, but I highly recommend Crafting Interpreters to anyone who wants to see what it is like writing a recursive decent parser and building a language out of it from scratch.
msangi
The second part (not finished yet) shows you how to build a VM and compile Lox (the toy language used in the book) too run on it.
runevault
Yup, actually planning to take a break after finishing part 1 to do something else, currently leaning towards getting more serious with Rust (to then use on part 2 or another similar style of tutorial)
msangi
That's great. I implement part 1 in Rust and I'll do the same for the second part
runevault
I started to do it in rust but was struggling too much trying to get the concepts straight AND get more comfortable with various things with Rust, decided to focus on one thing at a time so am doing phase 1 in c#. But as I want to build a compiler in Rust at some point I need to get comfortable with it in this context.
akdas
Related to the ray tracing materials, I taught a two-day crash course on writing a ray tracer. The materials are available online: https://avik-das.github.io/build-your-own-raytracer/
herbstein
That looks really interesting. One question I have. The Peter Shirley material creates something that has a realistic-ish look. The picture on your page, which I assume is representative of the built tracer, has a very unrealistic look. Do you know what causes this discrepancy?
nimblegorilla
What images are you comparing? One issue might be that three spheres floating in space just isn't something most people see in real life.
akdas
You're absolutely right. Normally, the surroundings provide a form of area lighting that my image doesn't have.
akdas
I use the Phong reflection model (https://en.wikipedia.org/wiki/Phong_reflection_model), which gives materials a distinct plasticky look. Peter's materials are either purely diffuse, purely reflective or glass-like (the third one is something I didn't cover in the course at all).

Peter also uses area lighting derived from the surroundings, whereas my implementation uses point lights that result in sharp specular highlights. There are no surroundings in my image, only the lights.

No doubt Peter's materials look great. I wish I had time to cover more in my course, but I wanted to make sure I covered the prerequisite vector math as well for students without strong mathematical backgrounds.

slaydemons
Request: Build a JVM (java virtual machine)
n_t
JVM might be much more involved project but here is a simple virtual machine - https://github.com/skx/simple.vm
stevekemp
That's my project, and it seems to be surprisingly popular, yet something I've never really received any feedback on.

At the time I wrote it I was modeling the opcodes on the Z80, but I guess I simplified once I'd got it working enough to make myself happy. (Lots of toy-virtual machines, of which this definitely is one, don't implement labels or "decompilers".)

msangi
http://craftinginterpreters.com/ part 2 (still a work in progress) will show you how to write a VM and compile a language to its bytecode.

It won't be the JVM but it could be a good start

pootsbook
This describes the purpose of the (paid) resource available at “The Great Code Club” [1].

Marc-André Cournoyer has put together several different projects: - 2D/3D Game - Database Engine - Virtual Machine - Backend + Frontend Framework - Neural Network - Language - Server - Real-Time Web Engine

Full Disclosure: I am a happy customer of Marc Andre’s “Owning Rails” [2] workshop. No other affiliation.

[1] http://www.greatcodeclub.com [2] http://owningrails.com

n_t
Very nice! These days think of something and someone somewhere has not just thought about it already but also making money out of it :)
reacweb
I think building a web server is a good way to learn (teach) HTTP protocol.
zaarn
I recommend the interpreter book [https://interpreterbook.com/], which isn't entirely free of charge however.

On the other hand, it was enough to get me off the ground for my lexer and write a parser that wasn't entirely dumb.

misternugget
As the author of the mentioned book I just want to add that I specifically wrote it because I'm also a huge fan of the technical writings the author is looking for: from the ground up, all code shown, no toys and shortcuts, capturing the essence in a few thousand lines.

And just FYI, the interpreter we build in the book ends up with ~3900 lines, including the full test suite.

n_t
Great stuff! Thanks for it.
zaarn
I think I'm atm at around 2500 lines, of which the parser and lexer is probably about 1200 lines. I mostly ~~stole~~ used the lexer from the book as inspiration for my own and then rolled a LISP-like language from there (which is rather easy given that a LISP AST is simple in nature)
dmytrish
Redox OS has `orbutils`, a bunch of simplistic GUI utilites written in Rust and somewhat cross-platform (it's possible to build and run them on other platforms) [0].

The most interesting and educational one for me is `browser`, which crudely shows web pages and is written in less than a thousand lines of Rust from scratch.

[0] https://github.com/redox-os/orbutils/tree/6764004b9f6f386af1...

nogridbag
Came across this website some time ago:

The Architecture of Open Source Applications

500 Lines or Less

http://aosabook.org/en/index.html

agentultra
I know some of the people behind that book. It's a very interesting take on the craft of programming. I highly recommend it.
n_t
It's great book. I liked it too.
acemarke
I've got a bunch of "Build a mini React" [0] and "Build a mini Redux" [1] articles referenced in my React/Redux links list. I particularly recommend the "Didact: a DIY guide to build your own React" post series [2] and "Build Yourself a Redux" [3].

[0] https://github.com/markerikson/react-redux-links/blob/master...

[1] https://github.com/markerikson/react-redux-links/blob/master...

[2] https://engineering.hexacta.com/didact-learning-how-react-wo...

[3] https://zapier.com/engineering/how-to-build-redux/

tarmstrong
"500 Lines or Less" is an entire book of articles just like this. Each chapter guides you through a small (500 loc or less) implementation of a common component (eg a web server). http://aosabook.org/en/index.html
zabana
Thank you so very much for this resource !
harlanji
I built a live streaming system with VoD constrained to RasPi+CDN with minimal code in a few weeks, fully FLOSS and offline capable. All design and hours of live hacking are online, comparison here: https://ispooge.com/2018-03-14-video-platforms-compared.html lmk if you want specifics — tutorials being produced on tinydatacenter.com. I am documenting as I learn, given I have no background in vid nor feature list in mind.

Tldr: all you need to make your own scalable Twitch without coding (but with patience). It’s a <2mo old so feedback appreciated, but my live vlog has been going strong for weeks and syndicating to 4+ other networks.