I love blog posts like this. You're not wrong in saying that the kernel is sort of this magical block box to most engineers (including me). I know how to use systemd and I know how to use bash and I know a few other things, but the kernel has always been "the kernel", and it's something I've never really tried to mess with. But you're right: ulimately the kernel is just a program. Yes, it's a big and important program that works at a lower level than I typically work at, but it's probably not something that is impossible for me to learn some basic stuff around.
I have had a bit of a dream of building a full desktop operating system around seL4 [1], with all drivers in user space and the guts fully verified in Isabelle, but learning about this level of code kind of feels like drinking from a firehose. I would like to port over something like xserver and XFCE and go from there, but I've never made a proper attempt because of how overwhelming it feels.
[1] I know about sculpt and Genode, and while those are interesting, not quite what I want.
> But you're right: ulimately the kernel is just a program.
Play a bit with user mode linux [1] the kernel becomes literally a linux program, that I believe you can even debug with gdb (hazy memory as I tried uml last time maybe a decade ago)
In theory you can also attach gdb to qemu running linux, but that's more complicated.
I would say something a little different. The kernel is a _library_ that has an init routine you can provide the function for. Or put another way, without the kernel your go program would have to have drivers statically compiled into it. This was the world of DOS, btw.
This is a really clean write up, but it is absolutely a happy path. I do feel the kernel is too big to be called a program. It is almost everything you want from comp sci class, router, scheduler, queue, memory manager. There are some interesting things that you have to handle if you do not run and OS and init on hardware e.g. handle signals, how do you shutdown, reap child process. I believe you are always better off with an init process and an OS.
the author's apparent epiphany is realizing that init is just a program. the kernel is, of course, software as well, but it does injustice to both "program" and "kernel" to lump them together.
Thank you for this quite perfect blog post (short, interesting, well written). One subject I would be interested in is what are all the parameters a kernel accepts
In the early days when the kernel was small (I used to build kernels and copy them to floppy disks, and boot Linux from there) the kernel was called 'vmlinux', and when compression was added after the kernel started to get bigger it became 'vmlinuz'. It was still possible to boot from 'vmlinux', and it may be possible today as well, for all I know.
Author here. It was a bit emotional seeing this on the front page.
My goal with this post and the whole (work in progress) series is to fill the gap between "here are the commands to do X" and "if you want to contribute to the kernel, you need to learn this" style books and tutorials.
I want something in between, for developers who just want a solid mental model of how Linux fits together.
The rough progression I have in mind is:
1. the Linux kernel as "just a program"
2. system calls as the kernel's API
3. files as resources manipulated through system calls, forming a consistent API
4. the filesystem hierarchy as a namespace system, not a direct map of disk layout
5. user/group IDs and permissions as the access control mechanism for resources (files)
6. processes, where all of the above comes together
I deliberately chose Go for the examples instead of C because I want this to be approachable to a broader audience of developers, while still being close enough to the OS to show what's really going on.
As a developer, this kind of understanding has been incredibly useful for me for writing better software, debugging complex issues with tools like strace and lsof, or the proc fs. I would like to help others to gain the same knowledge.
I guess also one of the points of using Go was the fact it has own memory management for obtaining memory pages it interacts only with the kernel.
I mean, had you used C, it would be better to compile it statically, otherwise you'd need to put also glibc and ld.so and what else into the initrd, I guess
The writing is really succinct and easy to follow.
One thing that could be improved is that the author could break down some of the commands, and explain what their arguments mean. For example:
> mknod rootfs/dev/console c 5 1
Depending on the reader's background, the args 'c', '5', and '1' can look arbitrary and not mean much. Of course, we can just look those up, and it doesn't make the article worse.
For anyone curious: "c" just means that it's a character device.
There is also "b" for block device (e.g. a disk, a partition, or even something like a loopback device) and "p" for FIFOs (similar to mkfifo).
The two numbers are just identifiers to specify the device, so in case of `5 1` it means the systems tty console, while `1 8` would mean "blocking random byte device" (mknod dev/random c 1 8)
I love that it's possible to boot a raw Linux kernel this way; I only learned about it very recently when working on a university project. It makes me want to fiddle around with it more and really understand the nuts and bolts of a modern Linux system and work out what actually is responsible for what and, crucially, when it happens.
Wow, what a nice and easily understandable explanation of an overcomplicated topic. This kind of teaching method is so much needed in software development.
That is: this seemed like the first 3 minutes of the first lecture on an freshman OS course, or similar in any book on systems. The complication you refer to - is it just from the clutter of adjacent words (EFI, grub, kmod maybe?)
Try to read a document/book on the linux boot process and it is VERY complicated if you actually want to know all the steps from POST to a tty login. You can strip some of it away but focusing on one path (UEFI vs BIOS) or just ignoring the instruction pointer movements.
I agree, little nuggets like this are valuable even if know it already.
Stupid question, but what does the default init program do? If I have a single application (say a game), can I just set up the file system, statically link my game and bundle it as an iso, rather than say containerising it?
Another cool way to show that 'the Linux kernel as "just a program"' is that you can also run the kernel as a regular binary without needing QEMU to emulate a full system:
> When the kernel starts it does not have all of the parts loaded that are needed to access the disks in the computer, so it needs a filesystem loaded into the memory called initramfs (Initial RAM filesystem).
The kernel might not have all the parts needed to mount the filesystem, especially on a modern Linux distro that supports a wide variety of hardware.
Initramfs exists so that parts of the boot logic can be handled in userspace. Part of this includes deciding which device drivers to load as kernel modules, using something like udev.
Another part is deciding which root filesystem to mount. The root FS might be on an LVM volume that needs to be configured with device-mapper, or unlocked with decrypt. Or it might be mounted over a network, which in turn requires IP configuration and authentication. You don't want the kernel to have those mechanisms hard-coded, so initramfs allows handling them in userspace.
But strictly speaking, you don't need any of that for a minimal system. You can boot without initramfs at all, as long as no special userspace setup is required. i.e., the root FS is a plain old disk partition specified on the kernel command line, and the correct drivers (e.g. for a SCSI/SATA hard drive) are already linked into the kernel.
I got close to this realization after learning barely enough U-Boot to launch my own bare metal program for the JH7110. I could never get into Linux From Scratch because it was more focused on getting an entire system working when I really just wanted to see how it spins up to get going.
Then at some point the other week I realized I could technically have a working Linux "system" with nothing more than a kernel and a dirt simple hello world program in /sbin/init.
I haven't had the time or inclination to scratch that itch but it's nice to see this article confirm it.
37 comments
[ 3.2 ms ] story [ 52.1 ms ] threadI have had a bit of a dream of building a full desktop operating system around seL4 [1], with all drivers in user space and the guts fully verified in Isabelle, but learning about this level of code kind of feels like drinking from a firehose. I would like to port over something like xserver and XFCE and go from there, but I've never made a proper attempt because of how overwhelming it feels.
[1] I know about sculpt and Genode, and while those are interesting, not quite what I want.
Play a bit with user mode linux [1] the kernel becomes literally a linux program, that I believe you can even debug with gdb (hazy memory as I tried uml last time maybe a decade ago)
In theory you can also attach gdb to qemu running linux, but that's more complicated.
[1] https://en.wikipedia.org/wiki/User-mode_Linux
I’d love a similarly styled part two that dives into making a slightly useful distro from “scratch” in go.
the author's apparent epiphany is realizing that init is just a program. the kernel is, of course, software as well, but it does injustice to both "program" and "kernel" to lump them together.
Thank you. I have always wondered that.
And updated domain: https://mustafaakin.dev/posts/2016-02-08-writing-my-own-init...
From https://news.ycombinator.com/item?id=41270425 re: "MiniBox, ultra small busybox without uncommon options":
> There's a pypi:SystemdUnitParser.
> docker-systemctl-replacement > systemctl3.py parses and schedules processes defined in systemd unit files: https://github.com/gdraheim/docker-systemctl-replacement/blo...
From a container2wasm issue about linux-wasm the other day: https://github.com/container2wasm/container2wasm/issues/550#... :
> [ uutils/uucore, uutils/coreutils, uutils/procps, uutils/util-linux, findutils, diffutils, toybox (C), rustybox, ]
My goal with this post and the whole (work in progress) series is to fill the gap between "here are the commands to do X" and "if you want to contribute to the kernel, you need to learn this" style books and tutorials.
I want something in between, for developers who just want a solid mental model of how Linux fits together.
The rough progression I have in mind is:
1. the Linux kernel as "just a program"
2. system calls as the kernel's API
3. files as resources manipulated through system calls, forming a consistent API
4. the filesystem hierarchy as a namespace system, not a direct map of disk layout
5. user/group IDs and permissions as the access control mechanism for resources (files)
6. processes, where all of the above comes together
I deliberately chose Go for the examples instead of C because I want this to be approachable to a broader audience of developers, while still being close enough to the OS to show what's really going on.
As a developer, this kind of understanding has been incredibly useful for me for writing better software, debugging complex issues with tools like strace and lsof, or the proc fs. I would like to help others to gain the same knowledge.
I guess also one of the points of using Go was the fact it has own memory management for obtaining memory pages it interacts only with the kernel.
I mean, had you used C, it would be better to compile it statically, otherwise you'd need to put also glibc and ld.so and what else into the initrd, I guess
One thing that could be improved is that the author could break down some of the commands, and explain what their arguments mean. For example:
> mknod rootfs/dev/console c 5 1
Depending on the reader's background, the args 'c', '5', and '1' can look arbitrary and not mean much. Of course, we can just look those up, and it doesn't make the article worse.
There is also "b" for block device (e.g. a disk, a partition, or even something like a loopback device) and "p" for FIFOs (similar to mkfifo).
The two numbers are just identifiers to specify the device, so in case of `5 1` it means the systems tty console, while `1 8` would mean "blocking random byte device" (mknod dev/random c 1 8)
1. /sbin/init
2. /etc/init
3. /bin/init
4. /bin/sh
It dropping you into a shell is a pretty neat little way to allow recovery if you somehow really borked your init
> Bailing out, you are on your own. Good luck.
https://unix.stackexchange.com/questions/96720
That is: this seemed like the first 3 minutes of the first lecture on an freshman OS course, or similar in any book on systems. The complication you refer to - is it just from the clutter of adjacent words (EFI, grub, kmod maybe?)
I agree, little nuggets like this are valuable even if know it already.
Purely academic.
- https://www.kernel.org/doc/html/v5.9/virt/uml/user_mode_linu...
> When the kernel starts it does not have all of the parts loaded that are needed to access the disks in the computer, so it needs a filesystem loaded into the memory called initramfs (Initial RAM filesystem).
The kernel might not have all the parts needed to mount the filesystem, especially on a modern Linux distro that supports a wide variety of hardware.
Initramfs exists so that parts of the boot logic can be handled in userspace. Part of this includes deciding which device drivers to load as kernel modules, using something like udev.
Another part is deciding which root filesystem to mount. The root FS might be on an LVM volume that needs to be configured with device-mapper, or unlocked with decrypt. Or it might be mounted over a network, which in turn requires IP configuration and authentication. You don't want the kernel to have those mechanisms hard-coded, so initramfs allows handling them in userspace.
But strictly speaking, you don't need any of that for a minimal system. You can boot without initramfs at all, as long as no special userspace setup is required. i.e., the root FS is a plain old disk partition specified on the kernel command line, and the correct drivers (e.g. for a SCSI/SATA hard drive) are already linked into the kernel.
maybe the audience is people who've never heard of init or thought about kernel vs userspace.
Then at some point the other week I realized I could technically have a working Linux "system" with nothing more than a kernel and a dirt simple hello world program in /sbin/init.
I haven't had the time or inclination to scratch that itch but it's nice to see this article confirm it.