I was about to comment the exact same thing. This book's bottom upwards approach is something that gave amazing insight on how well designed the Internet and computer networks are. I will forever be grateful to these guys for making free so a broke student like I was could afford that knowledge.
Potentially, once I cover the more basic topics. Part 1 will cover essentially a simplified Ethernet network (plus a few additions). Part 2 will cover the internet as it existed about 20 years ago. And part 3 will cover modern additions.
I’ve done some IOT work so it is something that interests me.
I did things like Nyquist/Shannon, things like Manchester encoding and stop bits, etc 20 years ago in an electronics module at university. Barely remember anything about them, and haven't used them at all. Some colleagues might as they deal with satellite clearance and uplinks, but I'm just nice simple IP, and it's not going to help me with my (presumably) BFD issue which is causing my BGP routes to one site to fail over in 30 seconds rather than a few milliseconds.
It's insanely rare in my industry to encounter anything that isn't ethernet, hell even ethernet with MTUs below 1500 are fairly rare outside of VPN provisions. Sure there are some, especially in the high end areas, but it's so rare and tends to eventually present as ethernet.
Far more use for almost everyone is an understanding of Ethernet, IP(4+6), TCP/UDP/ICMP and how routing works (the concepts more than the specifics - things like how traffic can route asymetrically, how packets can be sent on different routes, etc). You could include other protocols like LLDP and arp (maybe LACP as part of the different routes), mention about how a traceroute will hide information when routers either don't decrement MTUs, or just stick the entire packet in an MPLS packet and pass it through the network, mention how just because your traceroute shows 20% loss at hop 7 it doesn't mean hop 7 is bad, and again how the ttl expired response may be dealt with with a low priority, or travel on a completely separate network, etc.
I would think all of that is far more useful than knowing that you need Manchester to keep your clock or how you use PSK to increase the symbols through your medium
Wow really neat, nice work! I also love the "read on the web for free, buy a print/pdf copy" model. I rarely buy books now without being able to sample them first. I like your writing style, and I think it's well on the way to being a great book.
It might be a little early to start spreading the word though, with only 3 chapters ready so far. Hopefully when it's closer to being done you'll be able to get some attention from HN
Well done Seth. I hope you see it through. Writing in this area goes
stale suprisingly quickly so fresh takes are always welcome. It looks
like this could be pitched at 400 undergrad ""basic coms and nets"
(what's missing is a great text at 300 foundation level that doesn't
need too much math). You can still sell a lot of print books with the
pdf free online, so I hope that works out well for you.
Thank you! It's been fairly burnout resistant so far because I've been able to switch back and forth between writing, drawing, and building a custom static generator.
>It looks like this could be pitched at 400 undergrad ""basic coms and nets"
400 level undergrad is about the level of detail I'm aiming for (eventually). I've been using one of my old undergrad networking textbooks to do sanity checks.
>You can still sell a lot of print books with the pdf free online, so I hope that works out well for you.
I really like the idea of this. I look forward to reading what's finished now, as well as what will be released in the future!
I have a huge gap in knowledge for so many things that I interact with on a regular basis, but that I don't actually need to understand well in order to meet minimum job expectations.
This seems like a good book to address network basics. Sadly, I'm a huge procrastinator, but this seems especially important for me to address.
Books that have been in my backlog for years: Database Design for Mere Mortals, Designing Data-Intensive Applications, Building Secure and Reliable Systems, Writing a Compiler in Go, Crafting Interpreters.
All books that I know I need to read, but I just can't get around to it because there's no immediate reward for me, as someone who doesn't have to worry about any of these days in my day-to-day duties.
Thank you! Your backlog sounds like a fantastic list as well. I can wholeheartedly recommend Designing Data-Intensive Applications. I’m reading it a 2nd time for a book club at work, and I think it’s one of the best overviews of distributed systems out there.
That's interesting. I've given a talk various times over the years on "how the internet works", and when I later read Code I also noted the similarity in style. So no, not directly, but we apparently just have similar styles (not that I would place my book anywhere near Code).
It takes a surprising amount of insight to remove all the extraneous information from a drawing, bringing it to its essence. That's what makes your drawings, although appearing simple, so good.
P.S. Scott Adams has talked about using the same technique for the Dilbert comics.
That's high praise, thank you! To some extent the drawings have started guiding the writing when I originally thought it would work the other way around.
Nice work - so far this looks like a fun way to learn about how computers communicate with one another.
If I'm not mistaken, it looks like there's a typo under the section "Binary is better?": it reads "...then R is also number 5" when I think it should read "...then F is also number 5".
Awesome! Thanks for reporting the first typo! I'm currently deciding between building a report errata feature, or just making it public on GitHub and accepting PRs.
"Alice send the following marbles: Red, Green, Green, Red, Green, Red, Red–No, Yes, Yes, No, Yes, No, No. This translates to: nothing there, rock ahead, turn right, nothing there, rock ahead, turn left, nothing there."
I'm not sure that I'm qualified to write that book. But I do have an interest in game engines, so something with networking might be the perfect tangent for me one day.
I recently moved from a professional developer that didn't know how networks worked to one that gave an intro talk on HTTPS to a student group. I wish I had had a book like this to make that change faster.
One callout: a table of contents is needed for the online version. One that links directly to the chapters. And if one exists, it needs to be easier to find because I couldn't locate it.
>I recently moved from a professional developer that didn't know how networks worked to one that gave an intro talk on HTTPS to a student group.
That's awesome!
>I wish I had had a book like this to make that change faster.
That's the goal.
I definitely need a table of contents, but I was planning on adding it once I got a few more chapters done. Based on the comments though, I should probably add one quicker than that.
I really like how in depth you are going on foundational Layer 1 material so far. Example, I was so happy to see you discussing transceivers. I feel like mosts texts just give a cursory treatment to layer 1 and its hardware. I hope the next few chapters continue on with this layer 1 and hardware before moving up to Layer 2. I think it makes your book quite unique so far. Thanks for sharing.
Thanks for what you put together! Do you mind sharing who did the doodles? I'm working on my own free web book and I've been hoping to include illustrations like these.
Thanks! I started Feb 1st when I took paternity leave. My wife was already back at work by then, so I mostly worked on it while the baby was napping. I've been working on it in my spare time in the evening since going back to work.
I feel like walkie talkies are something all network engineers need to play with. Furthermore, remember that real world telephone and radio networks were in fact... Networks. (Called 'nets' in the radio world)
Ethernet, and 802.11, are both based on the concept of a singular channel shared by many individuals. When Alice talks, she says 'I'm Alice calling Bob, can you hear me Over'.
Alice is the source address. Bob is the destination address. Over is the stop code/stop bit signifying the end of frame. The payload is 'can you hear me'.
-------
This is everything up to layer 3. But if you wanted to send a radio/telegram across the country, you need one more layer, IP4 or the ARRL protocol (American Radio Relay League meets on the air at predestinated times to pass messages to each other on Ham radio frequencies. It's a human network)
I'm Alice, calling the Eastern Operators. I'd like you to relay a message to Bob for me. The message is 'Can you Hear me', Over.
This is a radio frame like all others, but an additional layer has been added. The radio operator has its own protocol built on top of the Frames.
------
Dave may overhear all of this chatter, but Dave knows to stay off the channel (otherwise he will cause interference). Only if Dave is addressed specifically would Dave respond (Ex: This is Alice, calling Dave. How are you doing? Over)
We can see that the operator is itself just another person on the network. The operator may have a message from the rest of the world.
This is the Eastern Operator calling Alice. I'm relaying a message from Bob. The message is 'I got your message Alice!'. Over.
This is how a router may take a message from the greater internet and pass it to your local network.
If you wanted to add a learning switch to the example, you'd also need to add the concept of multiple channels. Once a switch learns which channel the recipient is on, it would only send to that channel instead of broadcasting to everyone. It still starts to break down unless you torture the analogy a bit more.
There is no true radio analogue here in the sense of how OP was explaining it.
At least in a wireless context, CDMA and TDMA are flawed examples because, CDMA uses fancy math to split communications over multiple channels and TDMA cuts the channels up into timeslots. You don't get full bandwidth allotment and utilization.
To make a wireless switch, you'd need to have PTP optical/microwave/radio links which have highly directional antennas. This would give you full-bandwidth per user.
Even in a wired context, frame-based protocols inherently utilize TDMA (over multiple channels), just not on timescales you normally think about.
If I'm not mistaken, the radio example uses both frequency division and time division. You're on a particular channel (typically frequency, although AM exists too), and only one person can talk at a given instant, or else you get a collision which results in probably garbled output. (There's also space division at play, since radios are not infinitely powerful.)
If the relay hop is also done over radio, then it's most likely achieved by hopping to dedicated frequencies for backbone comms, ideally with a separate antenna per channel so we can listen on all of them simultaneously. (You could constantly sweep the frequency space, but it's simpler to just have multiple antennas / radios.)
A (modern) real-world example is that I can use my phone as a wifi hotspot. It relays packets to/from its upstream link (4G or even wifi again). (It also presumably behaves like a NAT, but that's an implementation detail.)
Well, there is already a term for that: "Bottom up". I used to think learning abstract design "Top down" is a better way to get good at design, but the "Bottom Up" way, by walking me through the implementation and design, sticks with me in the long run.
full duplex ethernet circuits (eg: a basic cheap 10Gbps 1310/LR carried over two $30 transceivers and a six foot fiber patch cable between two pieces of equipment) and a half-duplex TDD/CSMA regime like 802.11b/g/n/ac/ax are not the same thing.
yes they both enable ethernet connectivity between devices on fabrics of things that can see and arp each other.
the ethernet 10GbE might be shared by many individuals. but it's full speed both directions. one is half duplex, the other is not. one has contention on the shared media airtime, the other does not.
they are both possibly shared in the sense that many peoples' traffic may be carried across the 10GbE full duplex circuit, the the real world implementation is very different.
the real world implementations of FDD point to point (FCC part 101) or similar radio systems are closer to wired ethernet in concept, though with less capacity.
however I do agree with your general point that more people should familiarize themselves with the limitations and usage methodologies for half duplex IP data mediums and the best real world implementations.
I probably should have clarified that when I said "Ethernet", I meant classic 10 Mbps Ethernet that today's kids probably don't know about. The Ethernet hub acted like a bus: anything one connection said was broadcast to _ALL_ other connections.
Of course, today we have far faster point-to-point switches. But it helps to remember that the Ethernet-protocol was built in the days of old: half-duplex shared communication broadcast to everyone on your hub.
-------
Today's wired 1Gbps or 10Gbps ethernet are way way faster than half-duplex channels that 802.11 / Wireless technologies are forced to use (after all: the physics behind 802.11 and walkie talkies remains fundamentally the same)
There probably isn't a need for CSMA/CD systems in Ethernet anymore (since today's networks are all dedicated wires). But still, the CSMA/CD system exists for a reason, a legacy of the old "shared ether network communications" that gives rise to the name "Ethernet". (One Ether that everyone on the network can hear simultaneously, like a walkie talkie)
Raises hand. I remember discovering promiscuous mode, and a stern talking to from our network engineers. It was obvious I didn't know what the hell I was doing, so I got a great tutorial about the differences between hubs and switches.
I'd argue that the legacy of CSMA/CD still alive in ethernet is what makes it harder to implement efficient networks, as the switched network kinda tries to emulate the shared bus behaviour.
I admit that I'm not actually a network engineer at this point. I took a networking course back in college and what I've discussed so far is largely just the basics and how I've internalized my lessons from back then.
Looking at CSMA/CD more specifically, as well as "full-duplex Ethernet" (which are 100Mbit, 1Gbit, and 10Gbit modern Ethernet), it seems like CSMA/CD is no longer used in any full-duplex Ethernet Protocol.
I'm not 100% sure about that, since its outside the scope of my old studies. But it makes sense to me. It seems like "full duplex Ethernet" can only work with point-to-point communications. If you have point-to-point connections between exactly two stations, there's no need for carrier-sense (CS), multiple-access (MA), or collision-detection (CD) protocols at all.
CS: Wait for the current person on the walkie-talkie to say "over" before you yourself start talking.
MA: Walkie talkies: many people can access the same channel.
CD: If two people accidentally start talking at exactly the same time, they need to independently try to restart-talking. A random-number generator with exponential backoff is common for this sort of thing IIRC, though I'm not keen on the exact details.
The problem is that the switched network has to emulate behaviours that were free on CSMA/CD system. So we get MAC learning, Spanning-Tree Protocol, and ultimately if you want performant large scale ethernet switching (and not just terminate ethernet at point2point) you end up with complex software directly managing forwarding tables (all sorts of SDNs) or essentially turn ethernet into routed network (using IS-IS routing, two standards exists for implementing it, one of them - TRILL - is reportedly backbone of Cisco Nexus Fabric system).
There's a reason why another name of Layer 2 ethernet network is "broadcast area".
The book “CODE” uses exactly this pattern. It starts with two people talking to each other via cans connected with string, and works its way up to complex networks and processor architecture.
There is still so much of the tech stack that hasn't changed since this book was published in 2000 - TCP, IP, DNS, binary, logic gates, etc. But there's also much that has changed that could be written about - virtualisation, containers, wifi, fibre, cloud infrastructure, USB, GPUs.
I've gone ahead and bought the first edition, and then when the other is out, I'll add that to my collection too. Looking forward to reading it. No matter how much I learn, I find going over the basics again, reading introductions again, really helps to keep everything active up there. Let's face it, we learn so much but we also seem to lose so much as well.
I recently picked up CODE after seeing it recommended on HN a few months ago and it's really filled in the gaps in my knowledge of how computing technology evolved.
Many of us are amateur radio operators, which is basically walkie talkies. And furthermore, when we're all having a group discussing, it very much follows your last point: https://en.wikipedia.org/wiki/Amateur_radio_net
Wild guess: Due to overlap with previously flagged activity (the specific IPs currently pointed to by Cloudflare, or their system overly broadly marked gcp-us-west1-1.origin.onrender.com.cdn.cloudflare.net)
That's not a bad idea. Its already on GitHub (I showed my wife how to use git because she's doing the proofreading), so it would just require me making it public.
What would set this apart from something like Networking a Top-Down Approach? I read most of it and thought it went into great detail explaining the basics to the general audience. It also doesn't require any programming, well mostly!
It essentially takes the opposite approach to Networking a Top-Down Approach. I start out from a state where computers, and computer networks don't exist and build up from that point. Whereas A Top-Down Approach works backwards from the application layer.
>I read most of it and thought it went into great detail explaining the basics to the general audience.
Thank you!
>It also doesn't require any programming, well mostly!
Yep, I did have to throw in a bit of pseudocode for some of the microcontroller section.
137 comments
[ 3.5 ms ] story [ 179 ms ] threadhttps://book.systemsapproach.org
I'm definitely a bit guilty of this.
OP do you plan to cover stuff used in IOT? Treating HTTP as a magic black box kind of works fine until you have to consider MQTT, COAP etc.
I’ve done some IOT work so it is something that interests me.
I’ve struggled to find a book that doesn’t just hand wave layer 1 and assume layer 2 is ethernet everywhere.
It's not all that network related.
WiFi and xray transmissions is of course covered in most books though at least in terms of its capabilities.
What more do you want to know about level 1 and 2?
It's insanely rare in my industry to encounter anything that isn't ethernet, hell even ethernet with MTUs below 1500 are fairly rare outside of VPN provisions. Sure there are some, especially in the high end areas, but it's so rare and tends to eventually present as ethernet.
Far more use for almost everyone is an understanding of Ethernet, IP(4+6), TCP/UDP/ICMP and how routing works (the concepts more than the specifics - things like how traffic can route asymetrically, how packets can be sent on different routes, etc). You could include other protocols like LLDP and arp (maybe LACP as part of the different routes), mention about how a traceroute will hide information when routers either don't decrement MTUs, or just stick the entire packet in an MPLS packet and pass it through the network, mention how just because your traceroute shows 20% loss at hop 7 it doesn't mean hop 7 is bad, and again how the ttl expired response may be dealt with with a low priority, or travel on a completely separate network, etc.
I would think all of that is far more useful than knowing that you need Manchester to keep your clock or how you use PSK to increase the symbols through your medium
It might be a little early to start spreading the word though, with only 3 chapters ready so far. Hopefully when it's closer to being done you'll be able to get some attention from HN
Releasing early was mostly a choice to help motivate me. As well as to get early feedback and proofreaders.
>It looks like this could be pitched at 400 undergrad ""basic coms and nets"
400 level undergrad is about the level of detail I'm aiming for (eventually). I've been using one of my old undergrad networking textbooks to do sanity checks.
>You can still sell a lot of print books with the pdf free online, so I hope that works out well for you.
Here's hoping!
I have a huge gap in knowledge for so many things that I interact with on a regular basis, but that I don't actually need to understand well in order to meet minimum job expectations.
This seems like a good book to address network basics. Sadly, I'm a huge procrastinator, but this seems especially important for me to address.
Books that have been in my backlog for years: Database Design for Mere Mortals, Designing Data-Intensive Applications, Building Secure and Reliable Systems, Writing a Compiler in Go, Crafting Interpreters.
All books that I know I need to read, but I just can't get around to it because there's no immediate reward for me, as someone who doesn't have to worry about any of these days in my day-to-day duties.
P.S. Scott Adams has talked about using the same technique for the Dilbert comics.
If I'm not mistaken, it looks like there's a typo under the section "Binary is better?": it reads "...then R is also number 5" when I think it should read "...then F is also number 5".
Futurama had this figured out:
"One beep for yes, two beeps for no."
"Beep-beep"
"That means double yes!"
https://www.youtube.com/watch?v=fUKxgg4yPxs
The xkcd-style stick figures are particularly appreciated as a form of sugar to help the medicine go down.
I remember there was a good article on the theory, especially the rewind part.
I think a table of contents is warranted. Even if it just for 3 or 4 chapters.
One callout: a table of contents is needed for the online version. One that links directly to the chapters. And if one exists, it needs to be easier to find because I couldn't locate it.
That's awesome!
>I wish I had had a book like this to make that change faster.
That's the goal.
I definitely need a table of contents, but I was planning on adding it once I got a few more chapters done. Based on the comments though, I should probably add one quicker than that.
Ethernet, and 802.11, are both based on the concept of a singular channel shared by many individuals. When Alice talks, she says 'I'm Alice calling Bob, can you hear me Over'.
Alice is the source address. Bob is the destination address. Over is the stop code/stop bit signifying the end of frame. The payload is 'can you hear me'.
-------
This is everything up to layer 3. But if you wanted to send a radio/telegram across the country, you need one more layer, IP4 or the ARRL protocol (American Radio Relay League meets on the air at predestinated times to pass messages to each other on Ham radio frequencies. It's a human network)
I'm Alice, calling the Eastern Operators. I'd like you to relay a message to Bob for me. The message is 'Can you Hear me', Over.
This is a radio frame like all others, but an additional layer has been added. The radio operator has its own protocol built on top of the Frames.
------
Dave may overhear all of this chatter, but Dave knows to stay off the channel (otherwise he will cause interference). Only if Dave is addressed specifically would Dave respond (Ex: This is Alice, calling Dave. How are you doing? Over)
We can see that the operator is itself just another person on the network. The operator may have a message from the rest of the world.
This is the Eastern Operator calling Alice. I'm relaying a message from Bob. The message is 'I got your message Alice!'. Over.
This is how a router may take a message from the greater internet and pass it to your local network.
What would be the Bob-Alice-Dave equivalent to replacing the network hub with a switch?
So I guess cellphones instead of one-at-a-time walkie talkies, since everyone on the cellphone network has a unique connection.
At least in a wireless context, CDMA and TDMA are flawed examples because, CDMA uses fancy math to split communications over multiple channels and TDMA cuts the channels up into timeslots. You don't get full bandwidth allotment and utilization.
To make a wireless switch, you'd need to have PTP optical/microwave/radio links which have highly directional antennas. This would give you full-bandwidth per user.
If I'm not mistaken, the radio example uses both frequency division and time division. You're on a particular channel (typically frequency, although AM exists too), and only one person can talk at a given instant, or else you get a collision which results in probably garbled output. (There's also space division at play, since radios are not infinitely powerful.)
If the relay hop is also done over radio, then it's most likely achieved by hopping to dedicated frequencies for backbone comms, ideally with a separate antenna per channel so we can listen on all of them simultaneously. (You could constantly sweep the frequency space, but it's simpler to just have multiple antennas / radios.)
A (modern) real-world example is that I can use my phone as a wifi hotspot. It relays packets to/from its upstream link (4G or even wifi again). (It also presumably behaves like a NAT, but that's an implementation detail.)
When I write software I think of objects and messages in a similar way.
full duplex ethernet circuits (eg: a basic cheap 10Gbps 1310/LR carried over two $30 transceivers and a six foot fiber patch cable between two pieces of equipment) and a half-duplex TDD/CSMA regime like 802.11b/g/n/ac/ax are not the same thing.
yes they both enable ethernet connectivity between devices on fabrics of things that can see and arp each other.
the ethernet 10GbE might be shared by many individuals. but it's full speed both directions. one is half duplex, the other is not. one has contention on the shared media airtime, the other does not.
they are both possibly shared in the sense that many peoples' traffic may be carried across the 10GbE full duplex circuit, the the real world implementation is very different.
the real world implementations of FDD point to point (FCC part 101) or similar radio systems are closer to wired ethernet in concept, though with less capacity.
however I do agree with your general point that more people should familiarize themselves with the limitations and usage methodologies for half duplex IP data mediums and the best real world implementations.
I probably should have clarified that when I said "Ethernet", I meant classic 10 Mbps Ethernet that today's kids probably don't know about. The Ethernet hub acted like a bus: anything one connection said was broadcast to _ALL_ other connections.
Of course, today we have far faster point-to-point switches. But it helps to remember that the Ethernet-protocol was built in the days of old: half-duplex shared communication broadcast to everyone on your hub.
-------
Today's wired 1Gbps or 10Gbps ethernet are way way faster than half-duplex channels that 802.11 / Wireless technologies are forced to use (after all: the physics behind 802.11 and walkie talkies remains fundamentally the same)
There probably isn't a need for CSMA/CD systems in Ethernet anymore (since today's networks are all dedicated wires). But still, the CSMA/CD system exists for a reason, a legacy of the old "shared ether network communications" that gives rise to the name "Ethernet". (One Ether that everyone on the network can hear simultaneously, like a walkie talkie)
Looking at CSMA/CD more specifically, as well as "full-duplex Ethernet" (which are 100Mbit, 1Gbit, and 10Gbit modern Ethernet), it seems like CSMA/CD is no longer used in any full-duplex Ethernet Protocol.
I'm not 100% sure about that, since its outside the scope of my old studies. But it makes sense to me. It seems like "full duplex Ethernet" can only work with point-to-point communications. If you have point-to-point connections between exactly two stations, there's no need for carrier-sense (CS), multiple-access (MA), or collision-detection (CD) protocols at all.
CS: Wait for the current person on the walkie-talkie to say "over" before you yourself start talking.
MA: Walkie talkies: many people can access the same channel.
CD: If two people accidentally start talking at exactly the same time, they need to independently try to restart-talking. A random-number generator with exponential backoff is common for this sort of thing IIRC, though I'm not keen on the exact details.
There's a reason why another name of Layer 2 ethernet network is "broadcast area".
https://www.amazon.co.uk/Code-Language-Computer-Hardware-Sof...
I feel tempted to buy the current edition, but also to wait for the second.
There is still so much of the tech stack that hasn't changed since this book was published in 2000 - TCP, IP, DNS, binary, logic gates, etc. But there's also much that has changed that could be written about - virtualisation, containers, wifi, fibre, cloud infrastructure, USB, GPUs.
That way Dave knows to ignore the message if it doesn't start with his name (or "Everyone?" for broadcasts).
"This site is blocked due to a security threat."
>I read most of it and thought it went into great detail explaining the basics to the general audience.
Thank you!
>It also doesn't require any programming, well mostly!
Yep, I did have to throw in a bit of pseudocode for some of the microcontroller section.