Interesting, didn't hear from this system so far. Seems to be funded by the EU. Apparently it is written in pure Rust since 2020, and Andrew "bunnie" Huang seems to be involved.
It's not directly funded by the EU, it's funded by NLNet which is only in part funded by the EU. The goal is to collect money from large sources (e.g. EU) from relatively complex subsidiaries that are too big for small projects then dispatch and evaluate.
Thanks, of all the discussion on this HN comment page, your link (without even having to watch the video) finally answered my question of what the intended purpose is of this OS.
> Every Xous Server contains a central loop that receives a Message, matches the Message Opcode, and runs the corresponding rust code
Rust? Only Rust?
An OS has no business dictating implementation language. Inside my isolated microservice, I should be able to run anything I damn well please.
Rust's own safety guarantees are a red herring for security at this level, BTW, because you can't trust them over an IPC or system call boundary. The other process can just lie to you about being safe.
I'm a fan of microkernels and microservice models in general, but not if they sacrifice one of their core advantages: arms-length decoupling of implementation strategies through having isolated services communicate only through stable, versioned interfaces.
> A thread connects to a Server using a 128-bit ID. This ID may be textual if the server uses a well-known name that is exactly 16-bytes wide such as b"ticktimer-server" or b"xous-name-server", or it may be a random number generated by a TRNG.
What? This mechanism seems ripe for squatting attacks. How do I know I'm talking to the service I want to contact instead of somebody squatting the name? Using the name namespace for randomly generated IDs (binary!) or an ASCII name stuffed into the same bytes.
Better to give every object on the system its own unique unforgeable, unguessable ID and treat mapping from human-legible names to these strong IDs as its own service, one that can have namespace and authentication policies tailored to a given environment.
> since sending pages of memory is extremely cheap.
Depending on architecture, doing virtual address tricks ranges from expensive to exorbitant. Real-world systems doing bulk transfers over shared memory either rotate among pre-mapped buffers (Wayland, Surface/BufferQueue) or just have the kernel do one efficient scatter/gather memcpy into address space controlled by the recipient (Binder).
I'm not excited by this "lend" IPC primitive Xous has. Seems like more trouble than it's worth. You can add a queue of pre-mapped buffers on top as a separate service if you need it.
> Processes can allocate interrupts by calling the ClaimInterrupt call.
Good! It's about time more people write drivers as regular programs that treat IRQs like any other input event and less as magical things that for some ghastly reason must run with ultimate privileges just to do a DMA once in a while.
That said, just as a matter of elegance, I'd treat an interrupt literally like an regular input source and make it a device node on the FS, not some special kind of resource managed with its own system call.
In Linux terms, I should be able to open /dev/irq/5 and expect it to work like an eventfd. Isn't that elegant?
> ...memory will not be backed by a real page, and will only be allocated by the kernel once you access the page
Ugh. Contractual overcommit. Linux does overcommit too. It's an unfixable mistake. I'm disappointed to see a greenfield OS adopt the same strategy. Doubly so for an embedded system that might want precise control over commit charge.
See, in more mature virtual memory setups, we distinguish between reserving address space (which you do with mmap and such) and reserving allocated capacity (which we call "commit"). If you turn overcommit off on Linux (or use Windows at all) you get an elegant model where you can mmap(..., PROT_NONE) and not have your process "billed" for the memory in your allocated region -- but once you protect(..., PROT_WRITE), you can "charged" for that memory because after the mprotect returns, you're contractually permitted to write to that memory with the expectation you don't segfault or get some kind of "Opps. Just kidding. Don't have the memory after all!" signal.
> IncreaseHeap(usize, MemoryFlags) will increase a program's heap by the given amount
- One-way messages. You send, then, in a separate operation, you wait for a reply. This happens at each end. That means two extra trips through the scheduler and more time jitter. QNX has a blocking "MsgSend" which sends and waits for a reply. The scheduler transfers control to the receiving thread in the common case where the receiver is waiting, which behaves like a coroutine with bounded latency. It's a subtle point, but one of the reasons QNX is so well behaved about jitter.
- Interprocess communication by memory remapping instead of copying. This is high overhead for small messages, and at some fairly large size, becomes a win. Remapping pages means a lot of MMU and cache churn. Cost varies with the CPU and memory architecture. Mach worked that way, and the overhead was high. Not sure how expensive it is with modern MMUs. Do you have to stop other threads that might have access to the page about to be unmapped?
Unexpectedly wasn't asked: How it compares to Redox, another message passing microkernel system written in Rust? Also, what for embedded devices means? What specific features has that other microkernel systems don't, or just means is limited in scope*?
*Contrast to Redox that is meant to be general purpose but also offers an embedded-oriented minimal version.
My coarse understanding is monolithic kernels historically gained favor over microkernels for practical performance reasons bound to the cost of context switching.
I always felt there was opportunity to address that in order to have your cake and eat it too (eg. reserve registers for the kernel to avoid pop/push, maybe even dedicate a whole core and its state).
How fast / lightweight is context switching between userspace and kernel code in Xous?
Can you compile in such a way to avoid context switching altogether? What are the most significant overheads message-passing between services and where are is the lowest-hanging fruit for improving?
16 comments
[ 3.0 ms ] story [ 42.2 ms ] threadIs there a PDF version of the book (https://betrusted.io/xous-book/)?
Source : I have an NLNet funded project, so like Xous https://github.com/betrusted-io/xous-core?tab=readme-ov-file... I have such banners at the bottom of my repository.
> Every Xous Server contains a central loop that receives a Message, matches the Message Opcode, and runs the corresponding rust code
Rust? Only Rust?
An OS has no business dictating implementation language. Inside my isolated microservice, I should be able to run anything I damn well please.
Rust's own safety guarantees are a red herring for security at this level, BTW, because you can't trust them over an IPC or system call boundary. The other process can just lie to you about being safe.
I'm a fan of microkernels and microservice models in general, but not if they sacrifice one of their core advantages: arms-length decoupling of implementation strategies through having isolated services communicate only through stable, versioned interfaces.
> A thread connects to a Server using a 128-bit ID. This ID may be textual if the server uses a well-known name that is exactly 16-bytes wide such as b"ticktimer-server" or b"xous-name-server", or it may be a random number generated by a TRNG.
What? This mechanism seems ripe for squatting attacks. How do I know I'm talking to the service I want to contact instead of somebody squatting the name? Using the name namespace for randomly generated IDs (binary!) or an ASCII name stuffed into the same bytes.
Better to give every object on the system its own unique unforgeable, unguessable ID and treat mapping from human-legible names to these strong IDs as its own service, one that can have namespace and authentication policies tailored to a given environment.
> since sending pages of memory is extremely cheap.
Depending on architecture, doing virtual address tricks ranges from expensive to exorbitant. Real-world systems doing bulk transfers over shared memory either rotate among pre-mapped buffers (Wayland, Surface/BufferQueue) or just have the kernel do one efficient scatter/gather memcpy into address space controlled by the recipient (Binder).
I'm not excited by this "lend" IPC primitive Xous has. Seems like more trouble than it's worth. You can add a queue of pre-mapped buffers on top as a separate service if you need it.
> Processes can allocate interrupts by calling the ClaimInterrupt call.
Good! It's about time more people write drivers as regular programs that treat IRQs like any other input event and less as magical things that for some ghastly reason must run with ultimate privileges just to do a DMA once in a while.
That said, just as a matter of elegance, I'd treat an interrupt literally like an regular input source and make it a device node on the FS, not some special kind of resource managed with its own system call.
In Linux terms, I should be able to open /dev/irq/5 and expect it to work like an eventfd. Isn't that elegant?
> ...memory will not be backed by a real page, and will only be allocated by the kernel once you access the page
Ugh. Contractual overcommit. Linux does overcommit too. It's an unfixable mistake. I'm disappointed to see a greenfield OS adopt the same strategy. Doubly so for an embedded system that might want precise control over commit charge.
See, in more mature virtual memory setups, we distinguish between reserving address space (which you do with mmap and such) and reserving allocated capacity (which we call "commit"). If you turn overcommit off on Linux (or use Windows at all) you get an elegant model where you can mmap(..., PROT_NONE) and not have your process "billed" for the memory in your allocated region -- but once you protect(..., PROT_WRITE), you can "charged" for that memory because after the mprotect returns, you're contractually permitted to write to that memory with the expectation you don't segfault or get some kind of "Opps. Just kidding. Don't have the memory after all!" signal.
> IncreaseHeap(usize, MemoryFlags) will increase a program's heap by the given amount
What?? ...
Two surprising design decisions:
- One-way messages. You send, then, in a separate operation, you wait for a reply. This happens at each end. That means two extra trips through the scheduler and more time jitter. QNX has a blocking "MsgSend" which sends and waits for a reply. The scheduler transfers control to the receiving thread in the common case where the receiver is waiting, which behaves like a coroutine with bounded latency. It's a subtle point, but one of the reasons QNX is so well behaved about jitter.
- Interprocess communication by memory remapping instead of copying. This is high overhead for small messages, and at some fairly large size, becomes a win. Remapping pages means a lot of MMU and cache churn. Cost varies with the CPU and memory architecture. Mach worked that way, and the overhead was high. Not sure how expensive it is with modern MMUs. Do you have to stop other threads that might have access to the page about to be unmapped?
*Contrast to Redox that is meant to be general purpose but also offers an embedded-oriented minimal version.
So it is a kernel and can run on "hardware". On which "hardware", is left as an exercise for the user.
High resolution image: https://pbs.twimg.com/media/EMyz0mUXkAIyVMf.jpg?name=orig
I always felt there was opportunity to address that in order to have your cake and eat it too (eg. reserve registers for the kernel to avoid pop/push, maybe even dedicate a whole core and its state).
How fast / lightweight is context switching between userspace and kernel code in Xous?
Can you compile in such a way to avoid context switching altogether? What are the most significant overheads message-passing between services and where are is the lowest-hanging fruit for improving?