Correct me if I'm wrong, but I don't think that protected mode existed in old Unix systems and so was not delineated between user space and kernel space as described.
Protected mode and the separation of kernel and user space was only available since 80286 processors starting in 1982.
Virtual memory systems and protected memory certainly did exist before the 286, and Unix supported them well before 1982. Hell, the most popular CPU architecture for production Unix systems -- VAX -- was 32-bit and dates from the mid-1970s.
When Sun (?) wanted to implement their Unix with paging on the 68000 they had a problem: If you access a memory page that has been swapped out to disk, the OS has to (1) load the page from disk to RAM and (2) let the CPU repeat the instruction that caused the memory access. But the last step could not be properly done on the 68000 because it did not store its internal state completely when an (address) error happened (this was fixed in the 68010).
Their solution: Run two 68000 in parallel on the same code, one of them delayed by one instruction. When the first CPU triggers the page fault, the system can stop the second CPU before it reaches the instruction that caused the fault.
That's astounding. 68ks were not cheap. I assume the system then stalls for a couple of instructions to switch back to the first chip after the page fault completes? I have to assume they reached that solution after exhausting every other option.
On the other hand, this could also be used as a form of error checking. If CPU2 ever returns something different you know there was an error somewhere in the system and you can hard stop to prevent further data corruption.
> I have to assume they reached that solution after exhausting every other option
I assume that the 68000 was so powerful for its price (probably much cheaper than the existing mainframe CPUs) that it made this a viable solution. Or maybe the company had promised their customers a 68k-based Unix system (with the 68451 MMU) and the 68010 was delayed or too expensive and they had to find a quick solution? (I have no idea)
Btw, I was probably wrong about Sun being the company behind this. Apollo and MassComp have been mentioned in mailing lists.
The article describes a motherboard with both a soldered on 386 CPU, and a separate socket for another one. This was not a multiprocessor board, to make use of the socket you were supposed to disable the onboard CPU using a jumper.
But if you plugged in a similar enough CPU without setting the jumper, it appears that both CPUs ran at the same time. And since they had the same clock and they should have no difference in behavior, the system, at least as tested, ran fine.
Hah, that is amazing and baffling at the same time.
Elsewhere, you unfortunately sometimes have to plain emulate the instructions. VM86 mode on x86 is notorious for that; not so much when accessing memory but for all the vast privileged instructions that can trap into the monitor (e.g. CLI, IN, OUT, POPF...). VM86 Extensions ameliorated it only somewhat.
But VM86 also does not have much relevance anymore. It was a mode for running 16bit real mode software, and 64bit x86 CPUs don't support it anymore when in long mode.
I'm having a hard time understanding how this would actually work in practice.
CPUs act on data coming from eg the disk, memory, the network, etc. The results of instructions influences what the CPU will do next, especially so in the case of self-modifying code.
So, with a system that has a "chaser" CPU following a master CPU one instruction behind... how do you also impose a one-instruction delay on the CPU's view of the real world?!
Unless this is done, the chaser CPU is going to see real-world data and I/O that is out of sync with the instruction stream, isn't it?
I'm quite sure this problem was very elegantly solved, and I'm very curious+interested to find out what that solution was.
Would you agree that the whole TTY subsystem makes very little sense in this day and age? Why not let programs operate in raw mode by default and provide user-level library calls for input line buffering?
I don't know. It is one of the recurring questions in software development (or politics...) what should be centralised and what should be distributed. Job control might be one thing that we need a good default for anyway, so why not implement it in a central place?
That's not what happens with screen or tmux (or xterm, or sshd on the remote side of an ssh connection...). These allocate a pseudo terminal (PTY; which is a structure in the kernel) and connect to the master end. Using that they emulate a terminal, which has a virtual keyboard and a virtual screen. A shell (or another program) is started, with its input and output(s) connected to the slave end.
The virtual keyboard might be X11 key events in the case of xterm, or a network stream in the case of sshd, or simply the stdin downstream from another terminal in the case of tmux. That keyboard character stream is fed into the write fd of the master end of the PTY.
The virtual display is implemented by processing data read from the read fd of the master end of the PTY. That might involve drawing to an X11 window in the case of xterm, or sending back data over the network in the case of sshd, or writing to stdout connected to another terminal in the case of tmux.
The important part is this: The job control is still done in the kernel as part of the PTY implementation. That implementation is where for example a Ctrl+c (0x03 byte) is converted to a SIGINT which is then sent to the slave program.
Raw mode by itself is essentially useless for almost all programs; in order to do anything with it, you already need an entire surrounding collection of code to handle editing (even basic things like backspacing) and so on. But line-oriented input remains useful on its own without additional user-level code, again precisely because of the editing support the kernel already provides for you. Since you already need user level code to make any real use of raw mode, the user level code can also set things into (and out of) raw mode as you need it, so line mode remains a sensible default.
If anything things should move the other way, with the kernel providing more readline-style line editing in 'line mode' so that programs need to implement less of it themselves. But this would provoke all sorts of arguments over what line editing the kernel should support and so on, and then someday someone will suggest that the kernel should let you implement your own in-kernel line editing using eBPF (or WASM, if one leans that way), so people could push their preferred choice into the kernel when they log in or start up a shell on a new pty.
This really helps me understand how read() works as I am currently learning C programming. Didn't know that ctrl+D still sent everything before it in the buffer to the read call, and that a read of length zero and EOF was the same thing.
Ctrl-D is maybe the hardest part of TTYs to emulate if you want to bypass the accumulated legacy of the kernel-level TTY device API. It's very easy to make read() return 0 forever, but it's next-to-impossible to make read() return 0 exactly once, except for TTY slaves.
This is incredibly timely. I've been trying to figure out if it is possible to create a universal shim that would always give me readline-like behaviour while supplying input to an ongoing program.
Something like this:
./shim.sh | ./some_horrible_cli.rb
That way I could always rely on my linux shortcuts. Ie, no "^A^K" when I try to kill a line; it would just kill the line.
Th discussion of ssh is interesting and calls to mind the SUPDUP protocol, which moves all the character echoing, cooked mode, and even some local editing — including screen movement for Emacs!) to the local (user end) of the network connection. Might be worth reviving.
The fact that the kernel TTY driver is responsible for handling erase in cooked mode leads to a wrinkle: in order to do that correctly it needs to know if you're using a multibyte character set, and if so, which one (because a single erase must erase an entire multibyte codepoint).
Linux uses the (non-POSIX) IUTF8 termios(2) flag to indicate a terminal is using UTF8. If this is set on a terminal which isn't UTF-8, you'll see some odd results - for example, in a terminal using the single byte character set ISO-8859-1 but with IUTF8 set and in cooked mode, if you type:
áá
and then backspace once and hit enter, the process reading cooked mode will recieve "á\n" - the erase has eaten two ISO-8859-1 codepoints because it thought that they were a single UTF-8 codepoint.
I've never used a non-UTF-8 multibyte terminal (like BIG5 or SHIFT-JIS) but I don't think erase would work correctly in cooked mode on such terminals.
31 comments
[ 3.5 ms ] story [ 76.5 ms ] threadProtected mode and the separation of kernel and user space was only available since 80286 processors starting in 1982.
Refs:
https://en.wikipedia.org/wiki/User_space
https://en.wikipedia.org/wiki/Protected_mode
Have a look at the architectures of the PDP-11, VAX, 68000 and many other CPUs.
When Sun (?) wanted to implement their Unix with paging on the 68000 they had a problem: If you access a memory page that has been swapped out to disk, the OS has to (1) load the page from disk to RAM and (2) let the CPU repeat the instruction that caused the memory access. But the last step could not be properly done on the 68000 because it did not store its internal state completely when an (address) error happened (this was fixed in the 68010).
Their solution: Run two 68000 in parallel on the same code, one of them delayed by one instruction. When the first CPU triggers the page fault, the system can stop the second CPU before it reaches the instruction that caused the fault.
On the other hand, this could also be used as a form of error checking. If CPU2 ever returns something different you know there was an error somewhere in the system and you can hard stop to prevent further data corruption.
I assume that the 68000 was so powerful for its price (probably much cheaper than the existing mainframe CPUs) that it made this a viable solution. Or maybe the company had promised their customers a 68k-based Unix system (with the 68451 MMU) and the 68010 was delayed or too expensive and they had to find a quick solution? (I have no idea)
Btw, I was probably wrong about Sun being the company behind this. Apollo and MassComp have been mentioned in mailing lists.
A lot of companies wanted to use them instead of the Mickey Mouse Intel chips of the day, but the price point was too severe.
The article describes a motherboard with both a soldered on 386 CPU, and a separate socket for another one. This was not a multiprocessor board, to make use of the socket you were supposed to disable the onboard CPU using a jumper.
But if you plugged in a similar enough CPU without setting the jumper, it appears that both CPUs ran at the same time. And since they had the same clock and they should have no difference in behavior, the system, at least as tested, ran fine.
Elsewhere, you unfortunately sometimes have to plain emulate the instructions. VM86 mode on x86 is notorious for that; not so much when accessing memory but for all the vast privileged instructions that can trap into the monitor (e.g. CLI, IN, OUT, POPF...). VM86 Extensions ameliorated it only somewhat.
But VM86 also does not have much relevance anymore. It was a mode for running 16bit real mode software, and 64bit x86 CPUs don't support it anymore when in long mode.
CPUs act on data coming from eg the disk, memory, the network, etc. The results of instructions influences what the CPU will do next, especially so in the case of self-modifying code.
So, with a system that has a "chaser" CPU following a master CPU one instruction behind... how do you also impose a one-instruction delay on the CPU's view of the real world?!
Unless this is done, the chaser CPU is going to see real-world data and I/O that is out of sync with the instruction stream, isn't it?
I'm quite sure this problem was very elegantly solved, and I'm very curious+interested to find out what that solution was.
https://en.wikipedia.org/wiki/VAX#Virtual_memory_map
(although paging was in pdp-11 unix too, and neither was the 1st to have memory levels)
see also:
https://svnweb.freebsd.org/base/head/share/misc/bsd-family-t...
https://www.levenez.com/unix/
https://www.tuhs.org/
The 68K processors did not have a concept of protected mode. It just had user mode and supervisor mode[1]
[1]: https://retrocomputing.stackexchange.com/questions/2784/how-...
The virtual keyboard might be X11 key events in the case of xterm, or a network stream in the case of sshd, or simply the stdin downstream from another terminal in the case of tmux. That keyboard character stream is fed into the write fd of the master end of the PTY.
The virtual display is implemented by processing data read from the read fd of the master end of the PTY. That might involve drawing to an X11 window in the case of xterm, or sending back data over the network in the case of sshd, or writing to stdout connected to another terminal in the case of tmux.
The important part is this: The job control is still done in the kernel as part of the PTY implementation. That implementation is where for example a Ctrl+c (0x03 byte) is converted to a SIGINT which is then sent to the slave program.
If anything things should move the other way, with the kernel providing more readline-style line editing in 'line mode' so that programs need to implement less of it themselves. But this would provoke all sorts of arguments over what line editing the kernel should support and so on, and then someday someone will suggest that the kernel should let you implement your own in-kernel line editing using eBPF (or WASM, if one leans that way), so people could push their preferred choice into the kernel when they log in or start up a shell on a new pty.
(I'm the author of the linked-to entry.)
* https://groups.google.com/forum/#!msg/comp.sources.unix/Vejy...
Something like this:
./shim.sh | ./some_horrible_cli.rb
That way I could always rely on my linux shortcuts. Ie, no "^A^K" when I try to kill a line; it would just kill the line.
Is this possible?
Linux uses the (non-POSIX) IUTF8 termios(2) flag to indicate a terminal is using UTF8. If this is set on a terminal which isn't UTF-8, you'll see some odd results - for example, in a terminal using the single byte character set ISO-8859-1 but with IUTF8 set and in cooked mode, if you type:
and then backspace once and hit enter, the process reading cooked mode will recieve "á\n" - the erase has eaten two ISO-8859-1 codepoints because it thought that they were a single UTF-8 codepoint.I've never used a non-UTF-8 multibyte terminal (like BIG5 or SHIFT-JIS) but I don't think erase would work correctly in cooked mode on such terminals.