I wish they had more details up-front. Is this totally from scratch, or is it a piecemeal-replacement of some other Unix-like OS? Does it lean on any non-Rust components at all? If it's 100% bootstrapped from scratch and they already have a GUI like in the screenshot, that's extremely impressive and exciting.
Yeah, Redox is a completely new, from scratch, OS written in Rust. Ironically, even the libc is written in Rust. It (occasionally) runs on real hardware, and I believe they're pretty close to bootstrapping, although lack of a browser is a rather big obstacle
Rust is not supernatural. It addresses limited classes of bugs. Techniques to mitigate bugs in other languages have been developing rapidly. Bugs are not the only determinant of success.
Didn't realize it doesn't really run on hardware yet. Still impressive, though
Will they refuse to ship any userland code that isn't written in Rust (a browser, for example)? Writing a whole new browser from scratch in Rust would be almost as big a task as the OS itself. Practically speaking, it could make more sense to port an existing one over to their (already POSIX-compatible) environment.
No see my other comment https://news.ycombinator.com/item?id=24448512 . Not only do they support Unix interfaces, but you can even use a non-Redox-specicific package set with just a few tweaks.
It does run on real hardware, it's just very limited. Graphics card support is quite basic, it can't boot from a disk with a partition table (so the current ISO image uses a bootloader to load the filesystem into memory and emulates one), only the rtl8168d and e1000d Ethernet controllers are supported, and there's no USB HID or mass storage support so you need a PS/2 mouse & keyboard.
Indeed you could write an order of magnitude simpler browser, and also write the web pages it could render. Probably most people wouldn't bother using it since they want to read some other, legacy web pages (you know, to pay their bills, shop online, check the weather forecast, book their flights).
The analogy with operating systems is not silly, actually. Operating systems have indeed benefited/suffered from the network effect caused by "content" (programs). OSs were indeed becoming also feature-rich bahemoats that are hard to recreate faithfully enough in order to run that legacy stuff.
Operating system is imho a rather wide term. There is not that much to a simple operating system. They are mostly a scheduler. Manage some tasks, have the ability to switch between them - that's it. Things like FreeRTOS are not a lot more.
However when people refer to Windows or macOS as OS, they also include the whole collection of standard software as part of the OS. Control panels, browsers, GUI frameworks, etc. At that point it probably includes a browser
> However when people refer to Windows or macOS as OS, they also include the whole collection of standard software as part of the OS. Control panels, browsers, GUI frameworks, etc
I'd add hardware drivers to that list too. Many are of course written by 3rd parties, but to the average consumer they are "part of the OS".
There was once a programmer who was attached to the court of the warlord of Wu. The warlord asked the programmer: "Which is easier to design: an accounting package or an operating system?"
"An operating system," replied the programmer.
The warlord uttered an exclamation of disbelief. "Surely an accounting package is trivial next to the complexity of an operating system," he said.
"Not so," said the programmer, "When designing an accounting package, the programmer operates as a mediator between people having different ideas: how it must operate, how its reports must appear, and how it must conform to the tax laws. By contrast, an operating system is not limited by outside appearances. When designing an operating system, the programmer seeks the simplest harmony between machine and ideas. This is why an operating system is easier to design."
The warlord of Wu nodded and smiled. "That is all good and well, but which is easier to debug?"
Because browsers are now actually mature operating systems with many, many features and APIs.
I think a lot of people actually as much or more applications in their browser these days than normal applications. A lot of times they don't realize they are running an application though until it freezes the browser or pops up a window begging for money. But it's amazing how much code is downloaded for seemingly trivial applications sometimes.
But anyway if you take Chrome as an example, they are literally adding just about every OS feature including accessing USB devices.
This is cool, are there any downsides to calling Rust code from C? I guess it's basically a black box that behaves like a regular C library from the outside?
It is a regular .so (what I assume you mean by "C library"), not a "black box that looks like a regular .so".
There's no runtime wrapped around your code like with Go etc. You just need to add an attribute to tell the compiler not to mangle your names, similar to what's needed for C++ / C interop.
IIRC they're syscall implementation is ABI compatible with Linux so you should be able to run unmodified Linux binaries (assuming the syscall itself is implemented).
Is this a recent development? As far as I know they don't even support full POSIX. Simpler software can be ported by recompiling, yes, but I don't think they support Linux binaries, at least not yet.
I'm excited about Redox in particular, because of the lack of C.
But more broadly, I hope Nixpkgs (and eventually NixOS) can be a good way to promote kernel diversity. Every kernel having it's own distro is a huge waste, and I think the Nix in particular supports really good integration so the whole BSD "we're tighter-knit than a distro, don't call us that" argument shouldn't justify the duplicated effort.
> I think the Nix in particular supports really good integration so the whole BSD "we're tighter-knit than a distro, don't call us that" argument shouldn't justify the duplicated effort.
That's an inaccurate representation: The difference is that BSD evolves the whole system in lockstep, userspace and kernel at the same time. When userspace wants new capabilities like pledge(2), then the kernel is updated to expose it. When the kernel adds new features like routing domains, then userspace is updated to use it.
There's no coordination across a bunch of different projects: The same commit just changes things, across all the bits at once.
If I were to make a Nixified BSD, there's no requirement that I have to split things into multiple repos. I can continue to use a monorepo for the core userland and kernel.
If I want to add a feature to a widely used package, I can still vendor the package like BSD, or I can bump the kernel and package srcs in one commit, adding the feature in both, like Nixpkgs today. In no way am I now at the mercy of coordinating upstream devs---Nix allows any user to integrate anything they want, unilaterally, whether a monorepo is used ot not.
The benefit of Nixpkgs is not to replace the core of BSD, but to replace ports. Nobody, least of all the BSDs, has the time to bespoke integrate every single package. There is always a long tail of stuff that is budgeted minimally----this isn't Linux distro ideology, put simple economics. Nixpkgs allows multiple disparate groups to better cooperate on that long tail.
How mature are the network stack and FS? I see a ZFS inspired FS named TFS, very cool! I’m assuming it can run headless but see it’s got a GUI from the book.
This is more what I meant. It just seems like Redox is taking the lessons we've learned over years of development, and being willing to break compatibility in order to implement them.
That being said, I don't know much about OS/kernel development.
So is Google fuchsia - it is even capability based. (But Google is not cool here around HN circles, despite their contributions to open source and fostering one of good engineering environments, they aren't cool because they don't shout some Privacy buzzword like Apple.) Also it is not written in rust, which is webshit-favorite-of-the-month.
Redox is still Unix like, it moderately changes the file paradigm. Microkernel Unix clones are not new. They mention Minix as inspiration while there is no mention of L4 which did lot of work in microkernel performance.
At least it is capability based instead of yet another Unix clone. Not that any one of these are significantly better. But the rust crowd heralds redox as very innovative OS.
The performance problem has only gotten worse with time.
Modern CPUs have a rough time switching tasks. There are all those mitigations for meltdown and spectre. Even before that though, the situation with the TLB was ugly.
Limiting your data structures to ones that are friendly to message passing will really hurt you. Hacks to minimize the performance loss add complexity.
Fundamentally, glue isn't free. The parts may be simple, but the interactions are not.
Hardware IOMMU's have certainly made microkernels a lot more viable than they used to be in the past. You can actually place meaningful protection boundaries around core hardware drivers; they're no longer part of your inherent trust base, as they would be in a system where IO transfers can involve arbitrary memory.
Most embedded OSes for high integrity computing are microkernel based, then there is QNX, Minix running in legions of Intel CPUs, Android's workaround to have drivers as userspace processes with Android IPC as means to increase Android security and provide a proper driver ABI with Treble, and the ongoing push from Apple to move all kernel extensions into userspace, and then all those type 1 hypervisors.
Redox also has it's own non-standard shell that has been worked on for a while, called Ion. I don't believe it's entirely POSIX-compatible, and has some neat features of its own. That said, it's certainly possible someone could end up porting over Nushell to work on Redox, but I don't think that's currently on their roadmap.
The default shell is ion, a shell written in Rust and maintained as part of Redox. But I don't see it as important what the default is, so long as it is configurable.
We have ported bash and dash, so far. I think porting nushell wouldn't take much work
Hi Jeremy,
I'd love to hear what your thoughts are on Rust after building something like this. As someone who's relatively new to both Rust and OS development, I'm curious what some of the challenges have been around core data structures often found in kernels where ownership can be ambiguous or complex, and developing without the support of a runtime/standard library. Has Rust passed the acid test of OS development in your eyes?
core is very comprehensive and alloc adds dynamic allocation once your kernel supports a heap. The complicated data structures in kernels are much easier to write in Rust, in my opinion. And you can be certain of their consistency when you have multiple cores running kernel code because of the ownership model.
I prefer doing bare metal development in Rust if possible
Redox looks very interesting. It seems one good approach might be to force new apps and drivers for Redox itself, but provide a VM environment that would run sandboxed Linux and other OS's securely. (Similar to the Qube OS idea.)
That way Redox could offer a considerably improved API and app development environment without worrying about backward compatibility, while being useful as a daily driver in the meantime.
FreeBSD has a pretty broad linuxkpi layer for Linux drivers on FreeBSD as well. E.g., the Mellanox FreeBSD NIC drivers are really Linux drivers running on this framework.
What are the plans for bridging the ecosystem?
- Is there a Linux compatibility layer?
- Is there X11 support in Orbital?
- How difficult is porting Linux apps?
- I absolutely love the idea of using a memory-safe language for an OS, but realistically people will use whatever language they feel comfortable with for writing apps.
- Is it possible to write applications in something other than Rust?
Making the view releases button do something that isnt propping for a gitlab login, might entice people to make it past your homepage. Heck, add a "try it" button that redirects to the Book.
What are the things about Redox that aren't strictly Unix (or maybe POSIX) which you feel are marked improvements over Unix that folks with a general interest Unix & OS development should know about?
What alternative data structures do you have to use inside the kernel to provide good performance and satisfy the borrow checker without writing a lot of unsafe code?
Can anybody here do a qualitative comparison of the redox kernel and L4 (particularly SeL4)? I'd love to see how well some of the modern microkernels perform, but I think more importantly, how do they differ and what capabilities should we care about?
As a side question, how does kernel bypass networking work with a microkernel?
Bypass networking is generally more akin to how device drivers in microkernels work: a core trusted set of functionality that needs to be in the kernel is (e.g., programming the MMU or I/O MMU, scheduling) and all the device control and network stack stuff is in some process or task outside.
I have been curious about OS, particularly how a process for a command is handled through it's life cycle, and also things like how redirection works (for things like named pipes). Is there some resources on that topic? Searching for how OS works is very vague or too details in other aspects.
> particularly how a process for a command is handled through it's life cycle
fork(), execve(), waitpid() and friends
> also things like how redirection works
dup() ing file descriptors. dup2 is atomic alternative to separately closing a file descriptor and dup'ing to it.
xv6 book is freely available on MIT website - it teaches implementation of toy Unix like operating system.. as another commenter mentioned APUE is pretty popular too.
I apologize in advance for such a brazenly ignorant question: But why is it that this small team is able to produce a working microkernel, yet GNU Herd and (more recently) Google's Fuchsia have languished eternally as vaporware?
I don't think fuschia is vaporware. But I think the reason this seems more advanced than Hurd even being a lot younger is (also) because the concepts regarding microkernels have been expanded and solidified a lot since it started.
The manager asks the developer "how long is this project going to take you?" and the developer replies "2 weeks." The manager yells "That's unacceptable! How long will it take you if I give you another engineer?" to which the developer says "4 weeks." The manager asks again "Fine, take as many resources as you need. How long if I give you 100 engineers." to which the developer replies "it'll only take us 30 years to finish the design and after that it'll pretty much be smooth sailing!"
jackpot51 describes himself in his bio as a "BDFL of Redox OS" which means he's likely personally passionate and just does what he thinks is best. This will likely not lead to a better outcome than Herd and Fuchsia would lead to but it will lead to an outcome within our lifetimes.
In many cases, however, "working" is better than "planned"
You can use GNU HURD. I did so something like 20 years ago for kicks, then went back to Linux.
The reason stuff doesn't take off is that it never takes off. Implementing an OS is actually not hard. You can get pretty far as a 1-person project provided you have the time. Real world use and other people adopting your stuff is harder.
By "a working microkernel" you probably mean a working microkernel-based operating system.
If so, besides the hurd (and all the debian packages that build and run for it), there's Minix 3, HelenOS and Genode (specifically Sculpt) to look into.
There were used to be GNU/Hurd distros and Fuchsia (not a microkernel according to their FAQ; the reasoning being the many syscalls) was bootable with user interface within two years. Hurd development was/is slow because of architectural decisions they took, then Linux took off and GNU developers concentrated on providing good tools for it.
That may be a stupid question, but the comments here made me wonder:
Is the objective to have an OS written in Rust, which can support programs running in any language.
Or is the objective to have an OS written in Rust AND all the ecosystem in Rust too.
Im wondering because the former seems easier and reachable, but from the comments mentioning the port of numerous programs, I have the impression that it is the latter.
Bit curious that it doesn't seem to run on a Raspberry Pi. Would seem like an obvious thing to do at this point. Looked around and found some forum threads, but nothing installable.
107 comments
[ 2.9 ms ] story [ 164 ms ] threadEdit: There are more details in the Book https://doc.redox-os.org/book/ch01-02-what-is-redox.html
Will they refuse to ship any userland code that isn't written in Rust (a browser, for example)? Writing a whole new browser from scratch in Rust would be almost as big a task as the OS itself. Practically speaking, it could make more sense to port an existing one over to their (already POSIX-compatible) environment.
The analogy with operating systems is not silly, actually. Operating systems have indeed benefited/suffered from the network effect caused by "content" (programs). OSs were indeed becoming also feature-rich bahemoats that are hard to recreate faithfully enough in order to run that legacy stuff.
However when people refer to Windows or macOS as OS, they also include the whole collection of standard software as part of the OS. Control panels, browsers, GUI frameworks, etc. At that point it probably includes a browser
I'd add hardware drivers to that list too. Many are of course written by 3rd parties, but to the average consumer they are "part of the OS".
There was once a programmer who was attached to the court of the warlord of Wu. The warlord asked the programmer: "Which is easier to design: an accounting package or an operating system?"
"An operating system," replied the programmer.
The warlord uttered an exclamation of disbelief. "Surely an accounting package is trivial next to the complexity of an operating system," he said.
"Not so," said the programmer, "When designing an accounting package, the programmer operates as a mediator between people having different ideas: how it must operate, how its reports must appear, and how it must conform to the tax laws. By contrast, an operating system is not limited by outside appearances. When designing an operating system, the programmer seeks the simplest harmony between machine and ideas. This is why an operating system is easier to design."
The warlord of Wu nodded and smiled. "That is all good and well, but which is easier to debug?"
The programmer made no reply.
I think a lot of people actually as much or more applications in their browser these days than normal applications. A lot of times they don't realize they are running an application though until it freezes the browser or pops up a window begging for money. But it's amazing how much code is downloaded for seemingly trivial applications sometimes.
But anyway if you take Chrome as an example, they are literally adding just about every OS feature including accessing USB devices.
This is cool, are there any downsides to calling Rust code from C? I guess it's basically a black box that behaves like a regular C library from the outside?
There's no runtime wrapped around your code like with Go etc. You just need to add an attribute to tell the compiler not to mangle your names, similar to what's needed for C++ / C interop.
https://www.theregister.com/2019/11/29/after_four_years_rust...
https://linux.slashdot.org/story/19/11/30/2011231/rust-based...
https://doc.redox-os.org/book/ch01-06-how-redox-compares.htm... says they support the 31 most common syscalls (however that's defined).
2019 https://news.ycombinator.com/item?id=19478720
2018 https://news.ycombinator.com/item?id=18442390
2018 https://news.ycombinator.com/item?id=16198342
2017 https://news.ycombinator.com/item?id=15290607
2016 https://news.ycombinator.com/item?id=11318004
2016 https://news.ycombinator.com/item?id=12844539
2015 https://news.ycombinator.com/item?id=10295187
I'm excited about Redox in particular, because of the lack of C.
But more broadly, I hope Nixpkgs (and eventually NixOS) can be a good way to promote kernel diversity. Every kernel having it's own distro is a huge waste, and I think the Nix in particular supports really good integration so the whole BSD "we're tighter-knit than a distro, don't call us that" argument shouldn't justify the duplicated effort.
That's an inaccurate representation: The difference is that BSD evolves the whole system in lockstep, userspace and kernel at the same time. When userspace wants new capabilities like pledge(2), then the kernel is updated to expose it. When the kernel adds new features like routing domains, then userspace is updated to use it.
There's no coordination across a bunch of different projects: The same commit just changes things, across all the bits at once.
Nix doesn't offer that.
If I were to make a Nixified BSD, there's no requirement that I have to split things into multiple repos. I can continue to use a monorepo for the core userland and kernel.
If I want to add a feature to a widely used package, I can still vendor the package like BSD, or I can bump the kernel and package srcs in one commit, adding the feature in both, like Nixpkgs today. In no way am I now at the mercy of coordinating upstream devs---Nix allows any user to integrate anything they want, unilaterally, whether a monorepo is used ot not.
The benefit of Nixpkgs is not to replace the core of BSD, but to replace ports. Nobody, least of all the BSDs, has the time to bespoke integrate every single package. There is always a long tail of stuff that is budgeted minimally----this isn't Linux distro ideology, put simple economics. Nixpkgs allows multiple disparate groups to better cooperate on that long tail.
Also, in what way's does the project need help?
Edit: FWIW, I think this or something similar to it is the future of OS development.
I'll look forward to it.
> Right now writing drivers, porting software, and improving documentation are areas to help.
I'll take a look and see what I can help with.
Disagree. It is mostly a Unix clone in rust with few novel concepts. Not that it is not important, and no offense meant to developers.
Edit: removed italics tag.
That being said, I don't know much about OS/kernel development.
Redox is still Unix like, it moderately changes the file paradigm. Microkernel Unix clones are not new. They mention Minix as inspiration while there is no mention of L4 which did lot of work in microkernel performance.
A couple of weekends ago I was playing with seL4 on RISC-V and Rust under qemu, and it felt much better balanced overall than my recollections of LK.
https://fuchsia.googlesource.com/fuchsia/+/master/zircon/ker...
There used to exist a document about porting everything to C++, which I cannot find anymore.
Modern CPUs have a rough time switching tasks. There are all those mitigations for meltdown and spectre. Even before that though, the situation with the TLB was ugly.
Limiting your data structures to ones that are friendly to message passing will really hurt you. Hacks to minimize the performance loss add complexity.
Fundamentally, glue isn't free. The parts may be simple, but the interactions are not.
Hardly forgotten.
[1] https://github.com/redox-os/ion
We have ported bash and dash, so far. I think porting nushell wouldn't take much work
I prefer doing bare metal development in Rust if possible
That way Redox could offer a considerably improved API and app development environment without worrying about backward compatibility, while being useful as a daily driver in the meantime.
For example, something like NDISwrapper allows you to use Windows network drivers on Linux.
NetBSD's rump kernel drivers might be portable to userspace
https://en.wikipedia.org/wiki/MkLinux
https://genode.org/documentation/api/dde_kit_index
https://gitlab.redox-os.org/redox-os/redox/-/releases
From https://news.ycombinator.com/item?id=21839514 re: awesome-safety-critical https://awesome-safety-critical.readthedocs.io/en/latest/ :
> > Does Rust have a chance in mission-critical software? (currently Ada and proven C niches) https://www.reddit.com/r/rust/comments/5iv5j7/does_rust_have...
FWIU, Sealed Rust is in progress.
And there's also RustPython for the userspace.
As a side question, how does kernel bypass networking work with a microkernel?
fork(), execve(), waitpid() and friends
> also things like how redirection works
dup() ing file descriptors. dup2 is atomic alternative to separately closing a file descriptor and dup'ing to it.
xv6 book is freely available on MIT website - it teaches implementation of toy Unix like operating system.. as another commenter mentioned APUE is pretty popular too.
Hoping I'll be able to run redox-os as my daily driver in a few years.
>the project has been going since 2015
True, but Fuchsia started only a year later
>Fuchsia is a lot more than a microkernel, right?
Indeed, but that is also true of Redox!
jackpot51 describes himself in his bio as a "BDFL of Redox OS" which means he's likely personally passionate and just does what he thinks is best. This will likely not lead to a better outcome than Herd and Fuchsia would lead to but it will lead to an outcome within our lifetimes.
In many cases, however, "working" is better than "planned"
The reason stuff doesn't take off is that it never takes off. Implementing an OS is actually not hard. You can get pretty far as a 1-person project provided you have the time. Real world use and other people adopting your stuff is harder.
I can’t imagine how difficult it would have been to get support in 94, but 2020?!
If so, besides the hurd (and all the debian packages that build and run for it), there's Minix 3, HelenOS and Genode (specifically Sculpt) to look into.
>unix-like
You can't have your cake and eat it too.
"With POSIX compatibility" would have been a better description.
Im not following your contention.
And when Apple is finished with moving all kernel extensions to userspace, it might as well be considered as such.
By the way the extensions that were deprecated in Catalina, are now removed in Big Sur, as promised by 2019 roadmap.
So eventually, all extensions will be gone from the kernel.
https://en.wikipedia.org/wiki/Hybrid_kernel#:~:text=The%20Wi...
Is the objective to have an OS written in Rust, which can support programs running in any language.
Or is the objective to have an OS written in Rust AND all the ecosystem in Rust too.
Im wondering because the former seems easier and reachable, but from the comments mentioning the port of numerous programs, I have the impression that it is the latter.