When I read this I raised my eyebrow. I hardly think that Lisp (1958) can be the base of the operating system of the future. Their texts are full of bold statements about how easy and natural lisp is and how everything else sucks. Nevertheless it's interesting to look at their references:
I understand than in short, the security model is about how to make sure a process does not do more than what it's supposed to. In this case it's lisp environments that provide the necessary sandboxing.
That Lisp began in 1958 is mostly irrelevant. It (Common Lisp) is a fully-modern, Turing-complete language with a first-rate object system and supports all programming paradigms.
On top of that the implementations have all been regularly updated over the years and many are extremely mature and well-optimized.
Do you equate newness with quality, as seems to be common on HN?
Perhaps an OS written entirely in this month's JavaScript framework, deployed on Docker, using monads-as-a-service or whatever the hipsters have come up with lately would be more your cup of tea.
> Do you equate newness with quality, as seems to be common on HN?
When the link's thesis is "these OSes design years ago are old and therefore bad" it is hardly unreasonable to point out the language the author prefers is even older.
Because the UNIX based OS are based on PDP-11 architecture with the compromises required to fit in such systems.
Whereas already in the 60s, we had hardware systems like the Burroughs b5000 that could be fully programmed in Extended Algol and Espol, with zero Assembly.
The idea, as I see it, to base modern OSes in better older architectures that were lost to the mists of time.
Actually the mobile OSes, with their memory safe languages seem to already be moving into this direction.
>these OSes design years ago are old and therefore bad
That's not the link's thesis. Equally flippant but more correct would be to say "these OSes were written in memory-unsafe languages because hardware limitations demanded it back then, therefore they are bad"
If they came up with a good security model for applications, that would be a big step forward. Capabilities are a mechanism, not a model. Somebody has to decide who gets which capabilities. That mechanism needs to be understandable by the user. For example,
Privileges for a game:
- Can use full screen.
- Can read own content installed with app.
- Can read/write persistent storage associated with app.
- Can communicate with domain "thisgame.ea.com".
- Cannot run in background, access address book, ...
Word processor:
- Can access files mentioned in user file dialogs (file dialogs being run from a separate process).
- Can transmit requests to send mail to app "mail".
For any app which accesses information outside its own content and own persistent storage, the user should have the options "allow", "deny", and "fake". If a program asks for address book access and won't run without it, you can give it the "fake" option, which gives it access to a dummy address book. (I've seen something like this demoed for jailbroken Android phones.)
The biggest problem is the UNIX and Windows tradition that programs have all the rights of the user running them. Today, they should start out with the rights to read their own installed stuff, access their own private read/write prefs area, and use temp files. Anything more than that is at the discretion of the user.
So, who knows where it will go. Betting on a semi-functional language with prior high-security and high-productivity work is a better start than most projects like this, though.
I'm glad that alongside the philosophy of this being a FOSS endeavor, there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition.
Though I'm sure it's too early for outsiders to shape the PoC code, is it available anywhere for us all to follow along?
> there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition.
Is there the possibility that the OS code will be so mutable that it is impossible to predict if an application will run on any one installation? Will there be any posix-like standard to guarantee compatibility?
Does the language-based security model of 3L mean that all programs must be written in Lisp? Single-language systems tend to miss out on a lot of cross-platform software (which is typically written in or bootstrapped using C).
No. Almost any language we use today can run on 3L and transpilers are planned for the future of 3L. The security features will still be provided via the underlying lisp though. Technically the same security mechanism could work for most languages but it is much simpler, easier, and more natural to implement in lisp.
Even languages like C can be supported safely via memory emulation (although right now that would mean it runs quite slowly but likely, although no guarantee, it will run well enough in the future as hardware becomes more powerful).
I have always been curious about Lisps but the lack of static typing turns me off. Sure you can have a type system (typed racket) but it doesn't feel organic, also then why not use any other statically typed language (haskell, scala, ..). With all the hype (very likely well founded) surrounding Lisps, I feel like I am missing out.
Take a look at my own Haskell operating system project Hos (https://github.com/tathougies/hos). Written in Haskell 98 using the JHC compiler (a bit outdated, but makes it easy to generate ANSI C from Haskell sources). Currently supports multi-tasking, interrupts, paging, garbage collection. Has the beginnings of a simple user-space ATA driver.
Patches welcome. E-mail me at travis@athougies.net if you want help setting up a development environment!
Some of that is vastly simpler to do with a dynamic language.
"You can boot into a REPL (Read Evaluate Print Loop) and can inspect or rewrite any part of the OS as you are using it."
As far as I know, nobody has figured out a way to have your cake and eat it too with static typing. Some of the simple things that lisps have been doing for decades are very difficult if not impossible to do with statically compiled languages.
There's nothing in a REPL incompatible with static typing. Haskell, Swift, and Hack by FP all have interpreters with type provers.
But it's quite a lot of work to get acceptable OS performance if your entire system is interpreted or has to be JIT compiled all the time. People who do that tend to coincidentally decide they're so hardcore they don't even need JPEG support, video, or web browsing…
The way I see it is that the "killer feature" of Lisps is the macro system and macros fit more naturally in an untyped setting. In a way, macros are kind of a "compile-time eval" and eval is one of those functions that is very hard to statically type.
LISP is so powerful you can embed an ML in it while still getting its benefits for your use case. Incremental, per-function compilation, easy macros, and live updates were my favorite features. There's also been a ton of work running LISP's fast. Subsets even compiled to hardware in the past, including SHard (still around) and DDD.
You can have static typing with lisp if you want. There are many implementations of it actually. You can even detach the specifications of types from the code they refer to and specify the whole thing in lisp, no need to add any new syntax.
"Windows, Linux, Mac OS X, and the BSDs we have today are still built using the architecture developed when computing resources were limited even though it is no longer technically necessary."
Nonsense! Why would you waste any resources on infrastructure when you could use them for applications. Every cycle that your garbage collector runs is a cycle that the cpu could have been sleeping and saving battery life. This concept that we don't need to care about resources anymore is absolute poppycock.
The advantage isn't really in that resources can be wasted by adding a feature like garbage collection, but that if you are going to have features like that in your programs, building it in at the OS level is a better way to go about it, as it aggregates the resulting issues in one place, where it can be better managed(at a whole-system level).
>> Why would you waste any resources on infrastructure when you could use them for applications.
For the same reason we waste them on application frameworks instead of more interesting computations - because applications are written by humans, who take a long time to do it and are notoriously bad at it. Every cycle your garbage collector runs supports the existence of software that might not exist yet, or ever, without it.
I am all for advanced memory management in application frameworks. The OS however isn't something you're going to rewrite very often so work harder and squeeze out efficiency so that your apps get them.
OSes are constantly in development. Making development faster and easier could very well lead to a more efficient system. Faster and easier development makes it easier to test out new ideas and spend more time on optimization since it took less time to make the actual feature. Sure, if hardware never changed and the feature set never changed then you could just write the whole thing in assembly and make it as efficient as possible but that isn't reality. When it comes to software requirements and limitations are always changing so making it easier to adapt to those may in the end be the more efficient solution.
the same argument could be had against languages like Python, ruby, and JavaScript. making things easier to develop against often means that complex things exist that make things faster.
look at pypy for example: Python interpreter written in Python that's faster in many cases than cpython because they could build advanced features for optimising the underlying running code. writing an OS in C because efficiency seems like a load of hooey to me unless you have facts to back up your idea that for some reason the OS has to be the efficient thing.
software development comes down to 1 thing: delivering features. who cares if you need twice the servers to run your product of the alternative is you run out of runway and your project gets canned. hardware is cheap, engineering time is not
Ask yourself if the service you're trying to create would have been valuable, say, 4 years ago. Yes?
Okay, suppose you wrote it in C++ 4 years ago on four year old hardware? Is it still valuable? Yes?
Okay, now write it today in Node and run it on modern hardware. The hardware cost is the same, the service exists, and it was presumably cheaper to write and maintain in Node. Is it still valuable? Yes?
That's why it's reasonable to write apps in Node.js today: the performance and cost are the same as doing it in some "more efficient" language 4 years ago, except the cost to actually build the thing now in Node is (much?) lower.
If you were okay with the hardware costs for the service 4 years ago written in C++, you should be okay with the hardware costs for the service today written in Node.
You can use this argument to support "slow, inefficient" languages for many different kinds of projects (though not all, obviously). And I'm an efficiency nut.
> Nonsense! Why would you waste any resources on infrastructure when you could use them for applications. Every cycle that your garbage collector runs is a cycle that the cpu could have been sleeping and saving battery life. This concept that we don't need to care about resources anymore is absolute poppycock.
It isn't that simple. Depending on your program it can actually be slower to do manual memory management. malloc isn't free. In some cases allocating memory and performing GC is actually faster.
It's not that we don't have to care about resources at all just that we can sacrifice some of the resources to improve the situation for users and programmers. If that isn't your thing then feel free to write everything in assembly.
This isn't an either or. The reason your phone gets all day battery life is because the kernel was designed to run well on an i386. Let the app frameworks have cushy memory management but implement the OS in C.
> The reason your phone gets all day battery life is because the kernel was designed to run well on an i386
This is false. The reason your phone has the battery life it does is because the OS and libraries and applications have been written to allow the hardware of the phone to drop in to lower power states as much as possible. Unless the language you have written your OS in has some bizarre requirement to wake up all the hardware constantly then the language doesn't matter a ton when it comes to battery life. What matters is writing programs that wakeup the hardware as infrequently as possible.
Many times it is less resource intensive to do manual memory management, it just depends. The sacrifice in performance comes more from runtime features like type checking.
The only case that I know of when manual memory management is slower is short-running programs which exit before the first GC run, therefore removing the need to do any deallocation at all.
There is a market for a new OS, but security alone is not enough reason to make people switch.
Supporting Unicode would affect users a lot more.
I wish I saw an OS with good support for Chinese.
"Another change since the original OSes were initially being built is the need for security"
I do not know what's referred to as "original OSes" but many old operating systems had actually pretty decent security architecture. MULTICS and VMS, for example, and many others too.
It's true that some did. I was more referring to the history of most of the major OSes we use today and specifically when it came to the first microcomputers. Generally when they were first being used (and the OS was being expanded) security was not much of a concern.
Architecture designed in the 70's to run common stuff on terrible computers was necessarily bad. There was solid architecture from 1960's-1980's that was great for management, reliability, security, and so on. HW was too expensive. The alternatives flourished on minicomputers and personal computers with tons of software built on them. Legacy effects took over from there with nobody able or willing to re-engineer everything from scratch or even existing, complex code.
So, yes they suck. Yes superior architectures or languages of the time are worth copying or adopting over the knockoffs created due to hardware deficiencies. This is often even provable where one can look at problems that crop up in technology X caused by design point Y that's prevented by alternative design Z. And one of those alternatives was from 1961. ;)
OSes designed in the 70s aren't and weren't necessarily bad. It just depends on the circumstances. When they were written they fit the times quite well. But things have changed since then.
Lisps have also changed since it was first created. Unlike many languages lisp has evolved tremendously over the years and has benefited from a tons of language research. The lisps we have today are much more than we had in the 60s.
About time we have something modern that can do everything in that ad that's relevant today. I want my live-updating, live-fixing, per-function compiling, whole-system debugging, crash-resistant OS. If they could do it in the 80's then... you know... ;)
Genera (and other lisp OSes of the time) proved a lot of what 3L is building on. Unfortunately at the time the hardware required to run genera was expensive. Today the atmosphere is different.
I know (with software) to never say never, but to be quite frank, I'm content with the current Unix paradigm, and I haven't seen a compelling reason why any new OS would be fundamentally superior.
Entirely new OS would be a waste of resources, but some big player could donate few millions to boost the development of Wayland. What the markets lacks is a system with openness / hardware support of Linux combined with solid graphical layer of OS X. Or maybe AOSP could transform into true open source (culturally) project?
Too much network and filesystem data moves through an OS before the user application can work with it. As data bandwidth increases, this becomes a bottleneck. It would be great if an OS could push this out to userspace and focus on zero-copy stacks for network and disk.
3L does do this. You can share as much data between the OS and user applications as you want. The OS doesn't really mediate anything so there is no overhead from copying data or messages between processes or enforcing some security mechanism. The language runtime enforces the references that are available to some code that gets run but that's it and that's all that is needed. You really don't need a separate OS program facilitating everything to serve as an OS.
Yeah sorry my fault really. I'm working on improving the organization of the site and I haven't had near enough time to write everything down in an intelligible fashion. Most things are still in my notes or waiting to be extracted from various papers.
- OS's were designed for limited resources back in time. Now it's time to design something for unlimited resources.
- Attempting to retrofit a new security model to a current OS would require everything would be rewritten so we decided to rewrite an OS, which will require everything to be rewritten for it, including the OS this time.
- Sandboxing, local-scope FS access, user controlled permissions have never been done before, so we're doing those.
- Runtime analysis will give out better performance
- Compilers have always been hindered by OS design, so our new OS will break the barriers of compilers (finally! )
> Do I, as a user of the OS gain anything, or is it just easier for the kernel developers to not have to care about that?
Even if you don't gain anything directly you still gain significantly indirectly. Improvements in the speed and quality of kernel development means end users see new features quicker and bugs fixed sooner. It also makes it easier for CS researchers to try things out and make large advancements that can be easily applied to the OS.
> Improvements in the speed...of kernel development means end users see new features quicker
When is the last time you needed a feature that could only be provided by your kernel?
> It also makes it easier for CS researchers to try things out
How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)? Your premise is that life will be better because we won't have to deal with the ugliness of talking directly to hardware in C/C++/assembler, so what room does that leave for research?
Also I just want to tag on a question based on one of your reddit posts:
> One example is that 3L can save the entire system as you are using it. This means that if the power gets cut or the battery dies you can (instantly) reboot the system and it will be like you never lost power.
You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?
> When is the last time you needed a feature that could only be provided by your kernel?
Generally these are "unseen" features such as adding multicore support or adding support for a power saving feature on your graphics card.
> How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)
They are interested in a lot more than that. Check the 3L references page for a few examples. [0] Plus you can do many of those things with 3L.
> You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?
Yes. But it is certainly possible and has already been done to some degree. Genera, for example, did it and a number of other real-world and research OSes have done it. Of course the real challenge is making it as real-time as possible but getting it most of the way there isn't actually that difficult.
The Desktop are like unlimited resources, that is true to a certain extent. But the design has been switched to Phones in recent years, and they are limited. And we will move to Watch soon, which is even more limited resources.
Not to mention we are bound by Battery technology. In that sense, we are still very limited in resources.
Very true. When it comes to the resource intensive development features in 3L they can actually be disabled. But also this is partially a bet on the future, a bet that hardware resources will continue to become less constrained at all levels. Of course that is not for sure when it comes to that but it seems more likely than not that we will continue to advance in some ways. It's important though to start work now so that we are ready when the time comes.
Regarding batteries, the real issue there is how often you wake up the hardware from low power states. An OS like 3L could actually control that better than what we have today and give us better battery life.
Right now, not really. PreScheme currently compiles to C. Eventually the goal is to create an assembly backend and if that were combined with incremental compilation features you could manipulate it at runtime to some degree.
The only major disadvantage is it takes time to develop.
The advantage is it makes development of the OS easier and application debugging easier. It is also makes future proofing the OS more robust. A program that can rewrite or inspect other programs is very powerful, in the same way lisp macros are.
One concrete advantage of this is you could write a debugger as a user application, it wouldn't have to be integrated in to the language runtime.
> A program that can rewrite or inspect other programs is very powerful
And is a security nightmare. If I have an executeable, I have verified, I don't want the instructions to be altered in any way. In fact I prefer if it is impossible from a hardware level by marking them as executeable bits.
This still falls within the 3L security model though. A program can only do that if you, the user, allows it.
Also, 3L runs in two modes: development or secure. In secure mode you can completely disable manipulation of source code (or strip it out) and mark all code as read only and have the system verify the code hasn't been modified when it is run.
But how is that different than what we have today? Windows for instance already does this for system files, and could be expanded to user installed files as well.
You still need a debugging build, otherwise compiler optimizations will obscure the executeable to the point where debuggers cannot reliably debug it.
Stop waiting. Your reasons may be noble, some modesty about how nice you want the code to be when you finally release it to the world. But..we all know thats a flawed mentality. Open source it, now!
In summary, 3L drops explicit support for filesystems. Instead the system just provides a mapping of labels to storage blocks. From there those blocks can be utilized in any fashion you like. A filesystem library can be written to manage its set of blocks as it pleases. A database can skip the overhead of a filesystem and directly access the blocks to improve performance.
But yes, assuming you have imported the filesystem library (which has probably already been done assuming you are at the REPL) then you can process files as regular lists of lisp data (or in whatever language you want really). Not sure that is really that special though since you can already do that.
Keep going!
To make a new OS successful, there are too many difficulties. Existing competitors, acceptance of consumers and cost of shifting from the old to the new. We are happy to see a new OS. Hope it can go well!
91 comments
[ 8.3 ms ] story [ 146 ms ] threadhttp://3lproject.org/references
I understand than in short, the security model is about how to make sure a process does not do more than what it's supposed to. In this case it's lisp environments that provide the necessary sandboxing.
On top of that the implementations have all been regularly updated over the years and many are extremely mature and well-optimized.
Do you equate newness with quality, as seems to be common on HN?
Perhaps an OS written entirely in this month's JavaScript framework, deployed on Docker, using monads-as-a-service or whatever the hipsters have come up with lately would be more your cup of tea.
When the link's thesis is "these OSes design years ago are old and therefore bad" it is hardly unreasonable to point out the language the author prefers is even older.
Whereas already in the 60s, we had hardware systems like the Burroughs b5000 that could be fully programmed in Extended Algol and Espol, with zero Assembly.
The idea, as I see it, to base modern OSes in better older architectures that were lost to the mists of time.
Actually the mobile OSes, with their memory safe languages seem to already be moving into this direction.
That's not the link's thesis. Equally flippant but more correct would be to say "these OSes were written in memory-unsafe languages because hardware limitations demanded it back then, therefore they are bad"
Privileges for a game:
- Can use full screen.
- Can read own content installed with app.
- Can read/write persistent storage associated with app.
- Can communicate with domain "thisgame.ea.com".
- Cannot run in background, access address book, ...
Word processor:
- Can access files mentioned in user file dialogs (file dialogs being run from a separate process).
- Can transmit requests to send mail to app "mail".
For any app which accesses information outside its own content and own persistent storage, the user should have the options "allow", "deny", and "fake". If a program asks for address book access and won't run without it, you can give it the "fake" option, which gives it access to a dummy address book. (I've seen something like this demoed for jailbroken Android phones.)
The amount of data XPrivacy [1] can fake is impressive. It's a shame it isn't a built-in feature.
[1] https://github.com/M66B/XPrivacy#features
Security Kernel https://www.kyleisom.net/downloads/security_kernel.pdf
Formally-verified Scheme http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=881...
The features could certainly be worth it. Some aspects of the Genera experience, esp productivity & holistic debugging, still aren't in modern flows:
http://www.symbolics-dks.com/Genera-why-1.htm
So, who knows where it will go. Betting on a semi-functional language with prior high-security and high-productivity work is a better start than most projects like this, though.
I'm glad that alongside the philosophy of this being a FOSS endeavor, there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition.
Though I'm sure it's too early for outsiders to shape the PoC code, is it available anywhere for us all to follow along?
A mailing list to follow development will be available soon unfortunately I haven't had time to get it up and running yet.
edit: but check back to 3lproject.org soon or the kickstarter page. If you have kickstarted it I'll also make sure to send out an update.
Is there the possibility that the OS code will be so mutable that it is impossible to predict if an application will run on any one installation? Will there be any posix-like standard to guarantee compatibility?
No. Almost any language we use today can run on 3L and transpilers are planned for the future of 3L. The security features will still be provided via the underlying lisp though. Technically the same security mechanism could work for most languages but it is much simpler, easier, and more natural to implement in lisp.
Even languages like C can be supported safely via memory emulation (although right now that would mean it runs quite slowly but likely, although no guarantee, it will run well enough in the future as hardware becomes more powerful).
Take a look at my own Haskell operating system project Hos (https://github.com/tathougies/hos). Written in Haskell 98 using the JHC compiler (a bit outdated, but makes it easy to generate ANSI C from Haskell sources). Currently supports multi-tasking, interrupts, paging, garbage collection. Has the beginnings of a simple user-space ATA driver.
Patches welcome. E-mail me at travis@athougies.net if you want help setting up a development environment!
Some of that is vastly simpler to do with a dynamic language.
"You can boot into a REPL (Read Evaluate Print Loop) and can inspect or rewrite any part of the OS as you are using it."
As far as I know, nobody has figured out a way to have your cake and eat it too with static typing. Some of the simple things that lisps have been doing for decades are very difficult if not impossible to do with statically compiled languages.
But it's quite a lot of work to get acceptable OS performance if your entire system is interpreted or has to be JIT compiled all the time. People who do that tend to coincidentally decide they're so hardcore they don't even need JPEG support, video, or web browsing…
Example from sklogic that embeds ML in LISP https://github.com/combinatorylogic/mbase
An example implementation from CHICKEN Scheme: http://wiki.call-cc.org/man/4/Types
Nonsense! Why would you waste any resources on infrastructure when you could use them for applications. Every cycle that your garbage collector runs is a cycle that the cpu could have been sleeping and saving battery life. This concept that we don't need to care about resources anymore is absolute poppycock.
For the same reason we waste them on application frameworks instead of more interesting computations - because applications are written by humans, who take a long time to do it and are notoriously bad at it. Every cycle your garbage collector runs supports the existence of software that might not exist yet, or ever, without it.
look at pypy for example: Python interpreter written in Python that's faster in many cases than cpython because they could build advanced features for optimising the underlying running code. writing an OS in C because efficiency seems like a load of hooey to me unless you have facts to back up your idea that for some reason the OS has to be the efficient thing.
software development comes down to 1 thing: delivering features. who cares if you need twice the servers to run your product of the alternative is you run out of runway and your project gets canned. hardware is cheap, engineering time is not
I couldn't help but wince at this. I feel like this isn't a healthy attitude with respect to programming.
Ask yourself if the service you're trying to create would have been valuable, say, 4 years ago. Yes?
Okay, suppose you wrote it in C++ 4 years ago on four year old hardware? Is it still valuable? Yes?
Okay, now write it today in Node and run it on modern hardware. The hardware cost is the same, the service exists, and it was presumably cheaper to write and maintain in Node. Is it still valuable? Yes?
That's why it's reasonable to write apps in Node.js today: the performance and cost are the same as doing it in some "more efficient" language 4 years ago, except the cost to actually build the thing now in Node is (much?) lower.
If you were okay with the hardware costs for the service 4 years ago written in C++, you should be okay with the hardware costs for the service today written in Node.
You can use this argument to support "slow, inefficient" languages for many different kinds of projects (though not all, obviously). And I'm an efficiency nut.
It isn't that simple. Depending on your program it can actually be slower to do manual memory management. malloc isn't free. In some cases allocating memory and performing GC is actually faster.
It's not that we don't have to care about resources at all just that we can sacrifice some of the resources to improve the situation for users and programmers. If that isn't your thing then feel free to write everything in assembly.
This is false. The reason your phone has the battery life it does is because the OS and libraries and applications have been written to allow the hardware of the phone to drop in to lower power states as much as possible. Unless the language you have written your OS in has some bizarre requirement to wake up all the hardware constantly then the language doesn't matter a ton when it comes to battery life. What matters is writing programs that wakeup the hardware as infrequently as possible.
> we can sacrifice some of the resources
Which one is it?
I do not know what's referred to as "original OSes" but many old operating systems had actually pretty decent security architecture. MULTICS and VMS, for example, and many others too.
Old stuff is bad except when we like it, I guess.
[1] And quite dishonest to lump WinNT in with that...
So, yes they suck. Yes superior architectures or languages of the time are worth copying or adopting over the knockoffs created due to hardware deficiencies. This is often even provable where one can look at problems that crop up in technology X caused by design point Y that's prevented by alternative design Z. And one of those alternatives was from 1961. ;)
Lisps have also changed since it was first created. Unlike many languages lisp has evolved tremendously over the years and has benefited from a tons of language research. The lisps we have today are much more than we had in the 60s.
http://www.symbolics-dks.com/Genera-why-1.htm
About time we have something modern that can do everything in that ad that's relevant today. I want my live-updating, live-fixing, per-function compiling, whole-system debugging, crash-resistant OS. If they could do it in the 80's then... you know... ;)
Genera (and other lisp OSes of the time) proved a lot of what 3L is building on. Unfortunately at the time the hardware required to run genera was expensive. Today the atmosphere is different.
But I would absolutely love to be proven wrong...
http://www.catb.org/esr/writings/taoup/html/plan9.html
- OS's were designed for limited resources back in time. Now it's time to design something for unlimited resources.
- Attempting to retrofit a new security model to a current OS would require everything would be rewritten so we decided to rewrite an OS, which will require everything to be rewritten for it, including the OS this time.
- Sandboxing, local-scope FS access, user controlled permissions have never been done before, so we're doing those.
- Runtime analysis will give out better performance
- Compilers have always been hindered by OS design, so our new OS will break the barriers of compilers (finally! )
(and more!)
Do I, as a user of the OS gain anything, or is it just easier for the kernel developers to not have to care about that?
Even if you don't gain anything directly you still gain significantly indirectly. Improvements in the speed and quality of kernel development means end users see new features quicker and bugs fixed sooner. It also makes it easier for CS researchers to try things out and make large advancements that can be easily applied to the OS.
When is the last time you needed a feature that could only be provided by your kernel?
> It also makes it easier for CS researchers to try things out
How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)? Your premise is that life will be better because we won't have to deal with the ugliness of talking directly to hardware in C/C++/assembler, so what room does that leave for research?
Also I just want to tag on a question based on one of your reddit posts:
> One example is that 3L can save the entire system as you are using it. This means that if the power gets cut or the battery dies you can (instantly) reboot the system and it will be like you never lost power.
You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?
Generally these are "unseen" features such as adding multicore support or adding support for a power saving feature on your graphics card.
> How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)
They are interested in a lot more than that. Check the 3L references page for a few examples. [0] Plus you can do many of those things with 3L.
> You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?
Yes. But it is certainly possible and has already been done to some degree. Genera, for example, did it and a number of other real-world and research OSes have done it. Of course the real challenge is making it as real-time as possible but getting it most of the way there isn't actually that difficult.
[0] http://3lproject.org/references
Not to mention we are bound by Battery technology. In that sense, we are still very limited in resources.
Regarding batteries, the real issue there is how often you wake up the hardware from low power states. An OS like 3L could actually control that better than what we have today and give us better battery life.
The advantage is it makes development of the OS easier and application debugging easier. It is also makes future proofing the OS more robust. A program that can rewrite or inspect other programs is very powerful, in the same way lisp macros are.
One concrete advantage of this is you could write a debugger as a user application, it wouldn't have to be integrated in to the language runtime.
And is a security nightmare. If I have an executeable, I have verified, I don't want the instructions to be altered in any way. In fact I prefer if it is impossible from a hardware level by marking them as executeable bits.
Also, 3L runs in two modes: development or secure. In secure mode you can completely disable manipulation of source code (or strip it out) and mark all code as read only and have the system verify the code hasn't been modified when it is run.
You still need a debugging build, otherwise compiler optimizations will obscure the executeable to the point where debuggers cannot reliably debug it.
If your environment doesn't support C, how will you ever find code to run on it? Where is your GNU project?
Technically it could support C. One of the reasons it is based on Scheme though is that there is a lot of code already, code written in Scheme. :)
In summary, 3L drops explicit support for filesystems. Instead the system just provides a mapping of labels to storage blocks. From there those blocks can be utilized in any fashion you like. A filesystem library can be written to manage its set of blocks as it pleases. A database can skip the overhead of a filesystem and directly access the blocks to improve performance.
But yes, assuming you have imported the filesystem library (which has probably already been done assuming you are at the REPL) then you can process files as regular lists of lisp data (or in whatever language you want really). Not sure that is really that special though since you can already do that.
[0] http://3lproject.org/in-depth