Ask HN: How to get into OS/systems programming in 2023?
I graduated from college about decade+ ago, took few OS & systems programming courses, but didn't find them interesting back then. However, now, more than a decade later, I'm finding myself bored at my cookie cutter C#/Asp.net/SQL(+ cloud) job.
What would be your suggestions getting into OS/System programming in 2023 ?
For starters, I'm thinking about learning assembly, which I never learned.
41 comments
[ 3.3 ms ] story [ 101 ms ] threadhttps://pages.cs.wisc.edu/~remzi/OSTEP/
In the first step they ask you to build a simple shell with redirection and parallel execution which seems to be accessible to me (I don't have CS background).
MIT also has their own OS courses exposed to audiences:
https://pdos.csail.mit.edu/6.S081/2020/
The Wireguard drama in FreeBSD debunks this myth.
OSTEP, linked elsewhere in the thread is good, too.
[1] https://docs.freebsd.org/en/books/design-44bsd/
It's conceivable that they might have to write ASM at some point. However it will probably be very minimal and geared to a specific low level (hardware) goal.
There's still some value to "learning" ASM, or at least having worked with it if you want to write low level code, though it might be minimal. I wouldn't understand C and low level software without having worked with ASM at some point, even though I don't use it.
I probably wouldn't recommend it a good use of time.
Like C loops and ifs are made of jumps and compares in assembly.
<irony>goto is considered harmful</irony>
eg:
ARM (32) has LDR (32-bit), LDRH (16-bit), and LDRB (8-bit)
68K has MOVE.L (32-bit), MOVE.W (16-bit), MOVE.B (8-bit)
There are also some architectures (early Alpha for example) where memory can only be read in aligned 32-bit chunks or aligned 64-bit chunks. If you want to read 8 bits... well the compiler will generate a 32-bit read and then shift/mask the data to get the 8-bits that you care about.
If you ever do software vulnerability research, or reverse engineering it is also very important.
I know its a spicy take, and probably due to my reversing background, but I almost look at C as more of a macro language for ASM. In fact when I write in C, often times I think about what the compiler will produce. It also makes sure that when I write things, I am not doing it in an architecture specific way.
I just "ported" (nerfed the number of switches) the 1991 version of `ls.c` to modern Ubuntu and learned a lot of about `stat()`, `lstat()`, the `struct state`, file types in Linux and some programming tricks. It only has 1.2K LoC so it's easy to sort out the logic and write your own from scratch if needed.
* Take a course like Nand2Tetris (freely available online)
* Read a good OS book, e.g. The Design of the UNIX Operating System by Maurice Bach or any Andrew Tanenbaum book on the subject.
* Search for "unix systems programming" or "linux systems programming" and you will find some fantastic free books online.
* Read the free Linux Device Driver book. It is outdated but will still provide enough information to help you self navigate Linux driver code.
* Visit https://wiki.osdev.org/Expanded_Main_Page
* And as they say, "Read the source, Luke"; read the Linux source code. First you will have to identify which subsystem you are interested in, and then start browsing that.
It’s unfortunate Dennis Ritchie is no longer with us; it would be nice to have a third edition of K&R that covers modern C standards and has information about best practices for avoiding situations that can lead to insecure code.
Did not check whether it covers security issues.
http://knking.com/books/c2/index.html
It is still good because it gives information about general things which are applicable across any version.
BTW,just checked the meaning of outdated on Cambridge dictionary portal; outdated means "no longer useful or modern" When I used the word outdated I meant "no longer modern" :)
I've been programming embedded systems for the majority of my career and if you were to tell me that you just learned assembly, I'd wonder why you were wasting time like that. The last time I did more than glance at assembly language would have been around 1999.
If it's for a hobby, grab an arduino clone board and go for it! I'd suggest something ESP32-based though.
Straight out of Uni in 2001, I became an operating systems programmer bringing up a new platform and pretty much immediately started debugging C++ code almost exclusively in an asm debugger (it's what we had for an OS level debugger). I also started implementing capabilities provided by a hypervisor directly available via asm instruction because there wasn't a C construct for doing so).
After that I became a toolchain programmer (working mostly on new system libraries) and would write in C and Assembly language and debug mostly in Assembly language because, when you're working on system libraries and the toolchain for a new platform, you can't always trust the compiler, and the compiler can't always generate optimal code (or any code) based on the new instruction set. Oh, and you can't always trust the debugger either if it doesn't understand the instruction set.
While there I would work on 9 new platform bringups and would often need to stub in assembly language routines for capabilities that were in the architecture but not yet available in the toolchain. Other times, new system call routines, locking intrinsics, unwinders, exception handlers, and some optimized routines would require implementation in asm as well.
If you're only working on the non-platform part of an operating system, you might not need to learn assembly language, but if you touch platform code you will likely need to know it.
Compared to twenty years ago, there are significantly fewer operating systems which means fewer companies have a need for this type of work. If you're looking to get into this work I'd suggest looking at Apple, Samsung, Nvidia, Qualcomm, Google, Microsoft, Arm, and any Arm architecture licensee. It seems that the people with the most luck these days end up tinkering with Linux at university, do a Google Summer of Code project and score an internship with one of the above.
But those were systems where we abstracted away as much of the hardware as we could so everyone could focus on "business logic." The hardware interface is the part that's hardest to debug, so we try to put as little as possible into those layers.
Unfortunately, it sounds like that's exactly where you want to be working :-(
It's usually good to learn how to write ld linker scripts and how to make something bootable from grub.
Then, I would look Rust because it's easier to create correct no_std kernels.
EDIT: Link spew turned into a gist https://gist.github.com/5a1b94e8fde45e37c55c5a13d97c9b3f
[1] https://genode.org/