Actually Microsoft tried this idea themselves with WinFS. It was planned for Vista but eventually the project was stopped because it was too much effort. I could imagine this to be really useful if it was possible to add arbitrary custom attributes.
Does anyone have the story on why this project was actually halted? It reportedly sounded like it would have been a giant leap at the time, had it actually landed.
If the filesystem itself was modeled after a database (which it can be argued it is, somewhat), it's not much of a stretch for WinFS to have been a thing.
I was looking forward to seeing it in action based on reports of what it was.
File systems are databases in a small way. They use b-trees and such, but they don't enforce a rigorous global type system, don't enforce a table structure because everything has to have a schema, don't have the overhead of a query language to interpret every file access command, didn't ensure that all file accesses be ACID compliant, etc.
WinFS imposed all of those things and the associated overheads and much, much more every time you accessed anything.
It was very slow and resource hungry, and also very time consuming to develop things on. Imagine if you were only allowed to write or run applications that ran on top of Oracle. Even if you try to write a program that works on text files, those text files are stored as blobs in an Oracle database.
Isn't that the sort of problem you solve once, though? If you want a "this is just a stream of characters I can do POSIXy things to" interface, that's a library with fread(3) et al in the front, and a query for a single field in the database at the back.
It uses soup unions to manage removable storage, and worked really well for the 512k memory available.
These days I'd recommend an embedded SQLite database for program configuration and data - somewhat standard, easily recoverable, backups are easy, uniform access from a program's API etc.
Lastly, I'd say the ultimate OS would have a bit more than this surely - minimal formally verified TCB, Arrow datastructures, Managed (but not GC'd) memory, native scripting language that blurs the distinction between users and developers, fully scalable auto-generative but skinnable UI, just to name a few.
I guess the trick with all of this is a suitable language which forms the OS base (think LISP OSes, Micropython, Squeak, AOS, NewtonOS), you get that correct and the rest can be layered on top.
Those would be really cool improvements to see. And, in my experience, wise are those who have already built personal tooling to get some of this level of convenience functionality working for them. I'm still blown away by what can be accomplished by system scripts or simple apps _just_ in the realm of personal productivity.
> But it does not have to be this way. If a database engine is implemented as a core system service, along with simple tools to browse and modify database records, the gordian knot of system configuration files disappears. MacOS comes very close to this ideal, with ResEdit as this universal database editor.
> Or AS/400, which is more or less a SQL database.
People keep on saying that, but how really true is it? Many operating systems come with bundled relational databases–e.g. most Linux distributions come with more than one relational database implementation bundled. Does that make Linux "more or less a SQL database"? How deeply integrated is DB2/400 into the OS/400 kernel (or equivalent term, such as "System Licensed Internal Code")? There is a paucity of public information about the actual nature or depth of this integration.
I think an OS with a truly deeply integrated SQL database would expose things like a table/view of all files in the filesystem, a view of running OS processes, etc. As far as I am aware, OS/400 (or "IBM i" as they are calling it nowadays), doesn't have any such features.
If we are going to talk about NoSQL databases as an OS, we could also talk about early implementations of MUMPS and PICK, each of which was once upon a time delivered as a standalone operating system. (It's different now; at some point each were ported to run as packages on top of more mainstream operating systems such as Unix or VMS, and more recently Linux and Windows.)
Long time AS/400 programmer here. I think the confusion arises from mixing the terms "SQL database" and what IBM refers to as the "Single-level Store" [1]. The AS/400 has true orthogonal persistence, much like a database engine. But for a few specialized exceptions, there are no "files" anywhere on an AS/400 like on a typical operating system. Everything is an "object" (yes, extremely overloaded term) and there are specific ways to perform actions against objects.
To add to this confusion, IBM ported DB/2 to OS/400 atop the native OS/400 data object model. You can use SQL but that just gets compiled to native operations against what are known as "physical files" (confusing name, yes) and "logical files". Physical files are fixed-length record files with data in them. Logical files are indexes atop physical files. A PC analog to this would be FoxPro, Dbase, Paradox, Alpha, etc.
Many OS/400 programs access these files using the native model; not SQL. Tandem and HP Non-Stop work in very similar ways.
So, my impression, is many applications for AS/400 are written in RPG or COBOL, languages which in their AS/400 implementations treat "files" really no differently than they do on any other platform. Or, similarly, DB2/400 stores database tables as these single-level store objects underneath, but to a programmer writing SQL queries, it doesn't really make much difference – the experience is pretty similar to writing SQL queries for DB2 for z/OS, DB2 for LUW, or DB2 for VSE/VM (and what differences do exist are more due to the divergent code bases of the different products than due to the single-level store.)
I get the impression that S/38 and OS/400 have some really interesting concepts at the core of the OS, but it is questionable how well the higher levels layered on top leverage those concepts.
"Everything is an object" would be a lot more powerful if IBM let customers/ISVs define their own object types, when as far as I am aware they don't. Yet IBM will define dozens upon dozens of object types for all kinds of obscure requirements, many of which are no longer even relevant today [1][2].
I used to work with an AS/400, and never once thought of it as a database - sure, it had DB2 pre-installed, but I don't see how that makes the OS itself a database?
I only knew it only as the source of SQL-like data dumps. And people told me that it was more than an OS with DB2 as the default database. But maybe they were wrong.
ResEdit was an interesting idea. It suffered from a major problem - terrible data integrity. The original version had to work on floppies, with very slow seeks and writes. So it didn't reach an consistent state until you were all done and closed the resource fork of the file.
Attempts were made to use it as a database, but stuff was always getting corrupted.
The original MacOS suffered badly from being a cram job into 128KB with no hard drive or MMU. No CPU dispatcher, no memory protection, brittle ResEdit and TextEdit. The trouble was, that architecture persisted for 17 years, long after the hardware improved.
And it's tragic when you consider that LisaOS had multitasking, process separation, and I believe a hardware MMU.
The Mac grabbed most of the good UI aspects of the Lisa, but stripped out all the decent OS concepts. It worked to sell the first Macintoshes, but didn't scale out later, and by that time they'd killed off the Lisa.
Yeah, and the Lisa cost $10,000 and the Mac 128k was $2500. You can debate the merits of the hw/sw tradeoffs that got to those price points, but it was pretty clear at the time that Apple wasn't going to take over the world with the Lisa.
Tandem had a real database OS. They didn't have files. Just blobs in the database. Their distributed, redundant database system had proper ACID properties. It owned the disks; there was no file level below the database.
This made a lot of sense for a system intended for high reliability transaction servers and nothing else. Banks loved Tandems.
ElementryOS is heading that way, if you haven't heard of it yet. But the ecosystem is no where near close to MacOS. Sadly, for me, cause I mostly run Linux. :P
As an OS I agree, but the desktop experience leaves a lot to be desired.
I'm currently trying to figure out how to properly drive a 4K monitor with my laptop and not have it end up lagging. Supposedly hardware acceleration is enabled on Firefox, but even trying to play even a 1080p YouTube video drops frames. On Windows it is buttery smooth, so it's not a hardware issue. Don't even get me started about scaling...
I'd just like to interject for a moment. What you're referring to as Linux, is in fact, GNU/Linux, or as I've recently taken to calling it, GNU plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities and vital system components comprising a full OS as defined by POSIX.
I can sorta agree; a database with tables and folders would be a much more useful basis for an operating system. After all, most programs will read and parse small configuration files. With a database it becomes necessary.
Windows sorta does that with the Registry but A) the registry isn't the filesystem, B) the registry is a filesystem on top of NTFS and C) the registry sucks.
If the Registry had the power of,say, PostgreSQL to back it up, it would suck a lot less. Of course, for large files, you should still be able to rely on more conventional filesystems as well as for backwards compat.
It has no schema. Imagine if it had some sort of schema, declaring what sub-keys and values are allowed under each key. Imagine if the schema was self-documenting, with each key/value declaration in the schema had an associated description explaining what it was for.
Imagine if it had richer data types. For example, a "link" type, in which a value actually has the name of another key. If you try to delete the target, either it doesn't let you, or it sets the value to some sort of null value. (Basically, something like foreign keys in a relational database.) And an index to quickly find "back-references" (show me everything that points to this key.)
The registry should have included a database of installed packages/applications, and every key should have been marked with what package/application owns it, along with some sort of indexing to make it quick to find everything a package/application owns. Application uninstalls would have been a lot cleaner, and issues with apps leaving behind junk in the registry avoided.
Transactions: This was added in Windows Vista. But it could have been there from the beginning.
OTOH, remembering the registry was originally implemented in Windows 3.1, which had minimum system requirements of a 286 with 1MB of memory, maybe my suggestions above just wouldn't have been feasible.
I think it could, it just required a little more imagination.
NewtonOS was released in 1993 so around the same timeframe, lower memory requirements than 1mb and could do this stuff:
I always wonder where things would have gone if they had kept developing NewtonOS. It had so many interesting concepts that may have worked really well with a little more computing power.
I've been using osquery (https://github.com/osquery/osquery) for a while. It is neat and I can appreciate the idea of 'exposing OS interfaces as databases'.
osquery is cool. But, as far as I know, it doesn't expose the filesystem as a database, it is closer to /proc-as-a-database. (osquery can monitor specific files, in particular security-sensitive files, and expose events related to those files in SQL tables; but I don't think that facility is scalable from certain specific files to the entire filesystem.)
Indeed it isn't, in order to make file-level querying performant at all, you really need support for that at the filesystem level. Which is exactly what BFS, the BeOS filesystem had, and of course Haiku reimplements it: https://www.haiku-os.org/docs/userguide/en/queries.html
As that page describes, the "query" command (or its equivalent GUI) can be used to write filesystem queries, e.g.:
> Macintosh definitely stands out in this respect. Many similar functions within MacOS are accomplished by exactly the same action (e.g., removing is by dragging into trash, opening is by double-clicking).
Really bad example here IMO. All OSes have the concept of opening by double-click. And OSX is not consistent about dragging to trash, eg, can you drag an open browser window to trash? If yes, what does that do? Does it delete the loaded web page from your computer? Or does it leave bits of it in various caches? Or does it delete the browser from your computer? Or does it delete the content from wherever it is a stored (an email in gmail)?
I don't really know why you're getting downvoted. Plan9 is relevant to this space as it takes "everything is a file" to the logical conclusion in the same way that this takes "everything is a database" to the logical conclusion.
> The hierarchical organization of file systems manifests itself in nesting of directories (folders). On the other hand, directories are merely named views of a certain subset of files, selected according to some criteria. Thus, a directory can be thought of as a "database view," a named database query. It follows immediately that a file may appear in as many "directories" (views) as one wishes to. For example, one "folder" may show all files tagged as "sales reports", while another directory contains files modified within five days. Searching a file system and creating and populating a folder becomes therefore the same activity. Since saved views are database objects themselves, one can reference views within views if one so wishes. There is no required hierarchy however: one may create two views that refer to each other, or any other network of views that best suits the problem.
I built koios to solve this problem, but I didn't take the fuse approach because it seemed like a lot of extra work for the user, and puts databases first rather than files, which is un-UNIX.
First enumerate all of the constraints and guarantees your database provides. All the basic minimum services it provides for every access. Look at the memory it takes to provide those, and the processor utilisation overhead to perform all those checks, and services such as to guarantee ACID compliance, guarantee referential integrity, enforce a global type system, etc, etc.
Every single access of every single piece of data, for anything, will incur all of those overheads every time. There is no way to opt out or choose a different balance of guarantees or trade-offs without implementing them on top of the OS provided ones.
You're describing the fact that OS apis are typed, and check state for validity. That's already the case and OS system calls already incur those costs.
In a database, preparing a query compiles the access plan (ie all the above) on the server side. There's no reason it would be any less efficient than an OS call today.
Could in fact be more efficient, since you would probably be able to leverage set-based operations on the server (ie OS) side rather than iterating everything on the caller.
If OSes are already doing all these things, to the same degree required, and with the same overhead of a DB based OS, why do we need a DB based OS? What more is it giving us, and how is it providing that at no additional cost over the current model?
'On the server side' isn't magic. You don't get server side operations for free and these 'server side' operations would be occurring on the same computer.
Even my database doesn't incur all overheads for every operation. I don't know why an OS designed around the concept would. It's interface, not implementation.
It makes the conceptual model of the OS more complex. Filesystems with basic permissions are easy to understand becuse of the forced structure. IMO, ACLs are a bad idea for the same reason, they make it harder to see what is going on while not solving a problem I actually have. I guess they do solve problems that some people have so that makes them more attractive if you are one of those people but IMO there must be better ways to solve those problems.
Personally, I'd like to see version control and file compression integrated in my filesystem. I do not want my operating system to be distributed or rely on network access.
I also think a separation between administrative access to the full system and views of applications may be a good idea. It seems to me that the ideal OS would heavily restrict the file and network access of applications and there would be a separate administrative domain with full access but limited complexity. The operating system then manages the sharing of needed data between applications and network access that the user allows. I think having the system parse a standard configuration file format for applications would make sense in this model, but that configuration file can still just be text. Also having a sub-user permissions structure similar to the overall system users seems like a good idea to me. Many applications that users run are hostile and dealing with that requires major changes vs current desktop operating systems. Database vs. filesystem is a minor issue in comparison and personally I don't see the appeal of using a database.
Why not to trade a multitude of "custom" database managers for a single well-designed distributed database manager?
It's not clear whether the author is arguing that all existing OS data structures should be replaced with a single database system, or if they simply want all OS data to be capable of being queried/updated via a standard database style interface.
The former is simply hubris ("I know the perfect data structure for EVERYTHING!") while the latter is pretty much implementable as an interface layer on top of an existing OS.
The set of indexes are a form of data structure though, with all the state juggling and heuristics that usually go along with them (What access pattern requires this index? When do I have too many of them? What locks when I add/drop an index?). This adds complexity and unpredictability to the user.
I would argue that dropping and creating indexes is much less work than refactoring large parts of your codebase just to change a data structure.
Also, you can have multiple indexes on a table and the RDBMS ensures that they are always synchronized, which is something you won't achieve if you code the data structures yourself.
This is the key. You can even say that we only have a small amount of abstract "shapes" of data:
Scalars, list, tables(relations), tree/graph.
You can build generalized operations on them and it will work even if the internal structure (ie: list can be done on top of vectors or linked list or tables or tree) differ.
This is what I working on in my own little language (http://tablam.org). Certainly requiere specialize per structure and some stuff is hard to generalize but the idea hold.
There's another aspect to this that seems almost tangential as well. The author goes into the idea that program imports are directly referenced at the OS level. I suppose this replaces compiled system calls with some kind of foreign key reference?
The author mentions this removes the need for ids, but really it would just be hiding them in a non-serializable way.
Even things like USB are kind of database like. It's kind of silly that we have so many adhoc databases with their own query APIs (and different between every OS)
A lot of things are starting to work like operating systems for performance and/or security reasons. We might need to eliminate the distinction in our thinking to get better results on new systems. Quite a few programmers and researchers already have, making things like you describe. Others are building from hardware up to try to make their implementations more 1-to-1 with how hardware works.
Yes this has been true of major RDBMS for a long time. A conventional OS bootstraps Oracle (for example) and Oracle handles its own memory management, process scheduling, IPC, talks directly to storage, handles logins and interactive use, etc etc. Same with SQL Server. The AS/400 takes it to the extreme.
You raise a point - we have a bunch of small operating systems running on top of our existing operating systems. DBMSes, editors (emacs), browsers, and more. There's a great amount of inefficiences being added because everyone is re-inventing OSes and/or DBMSes at every layer.
I often wonder how much more efficient computers would be if we were capable of writing optimal machine code for whatever task we wanted to accomplish. No layers, just efficient code written at the lowest level for everything.
It obviously wouldn't be worth the effort, but how many times faster would our computers be?
I think it'd be possible to automate that at some level, generatively code just the minimum needed at the machine level for the task, and no extra fluff.
I remember people used to say, "Emacs is a nice OS, what it lacks is a good editor". While reading the Ultimate OS paper I was thinking about Emacs. It already exists.
Same. I've been reading through the book at project oberon and really like reading about the design decisions that went into it. I also think its UI is a great starting point to reason about a touch based IDE.
Already done! AOS was written in 2002, well before Android, and included a multithreaded ZUI by default (along with FAST native module compilation and dynamic linking).
Shame google didn't use it instead of Android, they had to invent Go to get back some of the AOS advantages.
— Development SDK based on Qt (other toolkits strongly restricted, gtk and gnome gone wrong way)
— Packaging based on something like APK from android or like DMG (no repos by default, only for base system maybe, but other methods of installing not restricted).
Snaps and flatpack are trash because their goal is so called security and isolation, when normal human being need just convenience (hate repos on desktop).
Yes, it's macOS model, but with linux you can have different filesystems.
I don’t think very many people agree with you. I appreciate Unix/Linux because they are fundamentally libertarian. I don’t have to suffer using some default crappy and bloated desktop environment (KDE certainly fits that definition), I get to choose (I personally like cwm).
You could certainly make a distribution in which using anything but Qt would be very very difficult. But who would want to use such a thing? Besides you, of course.
Moreover, your comment isn’t a cogent response to the post. Did you even read it? He’s talking about a new paradigm of OS, something fundamentally different. What you’re talking about is the same old same old, but worse.
It's all about the user experience. System should be friend and still enable the user to increasingly get more out of the computer. It also needs to be extremely responsive.
AmigaOS got a lot right back in the 80s. BeOS (and now Haiku as spiritual successor) took a lot from that (system kits vs amiga's default set of shared libraries, and the concept of datatypes) and added some concepts of its own.
Meantime, the mainstream systems have only gone backwards.
My dream of a general purpose OS is definitely an open source microkernel, multiserver RTOS with capabilities and an user experience that builds on those two systems. It is important for it to be an RTOS, as unbounded response times would make the system fail as a general purpose system and as a personal computer system.
The closest existing system would be Genode. It currently only meets the technical side unfortunately, but it could meet the user experience requirements with some work.
Being an RTOS is only necessary if your application has a requirement for real-time response and is also itself designd for bounded-response-time.
Ignoring drivers for the moment, your typical desktop application does not have a need for a bounded response time. Desktop applications typically do not fail due to a dialog box being popped up a few milliseconds late.
The underlying drivers, on the other hand, do tend to need bounded-response time, but this is typically performed by the use of the interrupt handling subsystem, whether directly handled or split into upper/lower.
Even non-RTOSs should be able to provide real-time response to drivers in this way and the trade-offs taken by an RTOS scheduler are not always appropriate or optimal for a desktop, user-facing system.
> Desktop applications typically do not fail due to a dialog box being popped up a few milliseconds late.
No they won't fail but a responsive user interface is critical. A process running on an RTOS won't complete any faster (probably the opposite), but displaying some sort of acknowledgement that a key has been pressed or a button has been clicked without any human perceptible delay is really nice. I'd love for the user interface portion of an OS to run with real time constraints.
Real-time response and a/an RTOS are not the same thing. An RTOS only allows processes with a known (or computable) runtime to run so that it may make guarantees about timing of execution. In some RTOSes scheduling may be entirely static and determined at compile time. This type of OS is used in critical systems (aircraft, hospital equipment, nuclear bombs). It's not really the type of system you'd want for a general purpose operating system.
> Desktop applications typically do not fail due to a dialog box being popped up a few milliseconds late.
Not a few milliseconds, but a few dozen seconds might be a different story. While it might be hard to predict or measure an exact number, your deadline is however long it takes for the user to equate it to "forever" and kill the process / leave the website / uninstall the app / etc..
A 'real time' OS does not mean fast. It is simply that the OS can promise an application/driver that it will get the processing time that it requests. e.g. a nuclear power station controller might use a RTOS to guarantee that the emergency shutdown process can be completed in under a certain number of seconds, regardless of whatever else is going on.
In general, being able to make these kind of promises means that the OS has to be pretty conservative about the code it allows to run, and a 'best effort' OS or some 'soft' RTOS is probably going to be faster or more efficient.
In terms of UI responsiveness, I'd bet that crappy software running on a 'hard' RTOS can still manage to feel sluggish, because the UI delays probably aren't actually in the OS itself.
There is some benefit to real-time ability in an OS. Mainly, to prevent one application from freezing up the system. That's happened to me both on Windows and Linux. Took work to wrestle it back into my control again.
I remember people who used the QNX demo disk on old machines describing how they could do big compiles in the background with the system still responsive to input. I'm guessing the parts interacting with the user had higher priority.
Real-time might help in another way: mitigating covert channels. Gotta know the timing of secret-handling code. Then, make the observed timing fixed-length or random. Don't always need the timing to do that, though. For instance, the separation kernels just need timing for the partitions with whole thing stopping regardless of where code is at.
One last benefit might be easier performance monitoring and diagnostics. If it's predictable, one might be able to assign a range or profile to it. Then, raise alarm if anything goes out of profile. Just speculating here: never built one.
A great many people work with audio or want to work with audio, and Windows and OSX are lightyears ahead of Linux in terms of ease of use and overall quality.
You'd think the PulseAudio fiasco would have improved the situation, but I believe the developers thought as you did, and so built a system that's woefully encumbered with latency. Jack improves the situation WRT latency, but it comes at the cost of an altogether brittle and broken integration with PA, and thus with all major Linux desktops.
Bounded response times may not be needed for your word processor, but there's plenty of folks out there who would be better served by it. Hell, I'd love a snappier editor on account of predictable response times to keypresses.
> You'd think the PulseAudio fiasco would have improved the situation
PA is in rather good shape now & it has capabilities its predecessors didn't. But it's very hard to get people to update their notions about something once it sets in.
No matter how much Microsoft tries, Windows will always be 'insecure, virus-ridden', macOS will always be super stable, beautiful, no matter how much Apple screws up & Linux will always be unusable on the desktop no matter how many decades I'll spend enjoying it on the desktop at home and at work.
Sadly, it seems a person's mind tends to naturally lean conservative and it takes a herculean effort to regularly update one's preconceived notions.
If you want realtime, there's JACK. PA is intended to replace/supplement ALSA, it has features like network transparency, per app volume control, EQ, virtual outputs etc. Not every SW has to have every feature, but PA has useful features ALSA doesn't.
In my experience, jack only manages to avoid xruns on reasonably small yet still too large (5ms) buffers when running as SCHED_FIFO or SCHED_RR, and only on linux-rt.
Patch sets like -ck simply don't help with this.
Try running cyclictest from rt-tests for a while to see the true horror of Linux behavior.
>Being an RTOS is only necessary if your application has a requirement for real-time response
Or, in other words, the operating system is not general purpose, because it is not suitable for these uses.
Do note these uses do include the likes of audio. If there's no way to guarantee latency, then a spike will surely eventually cause an overrun, which will be perceived as an audio hiccup. Sometimes, this is just an annoyance. On a live performance or in pro audio work, it can be fatal.
A good microkernel will allow mixed criticality tasks to share the system. A lot of recent research in the topic has brought SeL4 to this point.
> My dream of a general purpose OS is definitely an open source microkernel, multiserver RTOS with capabilities and an user experience that builds on those two systems. It is important for it to be an RTOS, as unbounded response times would make the system fail as a general purpose system and as a personal computer system.
This sounds more like a complete version of the Fuchsia Operating System, which has both technical and is more of a user-facing OS. That probably has more of a chance of meeting these requirements at the rate that it is being developed.
When are we going to truq u? You are almost as high profile here as Thomas ptazec and the venerably over full idlewords. I'd not truq idlewords though come out with it. Tyingq has to be a Maine trucker. Let's for the good of all of us.
I see it as
1) Thomas ptachech
2) idlewords you ass inferior trastard piece of skit
And never less 0) tryingq which is you.
Are you so ascetic? Don't tell us you got a text buddy on the side
Just a general question: Do you think there is a market for a new operating system? The cost of building something better or equal to mainstream systems seem too high to even consider challenging the status quo.
I think there is, but it'd need to be very different to existing operating systems to justify itself, and in ways that other OS's can't simply adopt for themselves. In practice that means radical architectural change, and even then you'd re-use a lot of code.
For such a project you don't really want to just adopt existing ideas and implement them. I don't really understand why Google is doing Fuschia for this reason: architecturally it's nothing special. Is the GPLd Linux so bad? It took them this far.
If I were to do a new OS I'd explore ideas like all software running on a single language-level VM with a unified compiler (e.g. a JVM), a new take on software distribution ... maybe fully P2P, rethinking filesystems and the shell, making it radically simpler to administer than Linux, etc. You'd need to pick a whole lot of fights and make a lot of controversial decisions to justify such a lot of work: you'd need to be right about things other people are wrong about, not just once but multiple times. Then maybe if people see it working well they'd join you and port existing software, albeit, the porting process would inevitably involve some rewriting or even deeper changes, as if you can just run existing software and get all the benefits you probably didn't change anything important.
Build a new operating system, or build an operating environment that runs on top of an existing operating system, like how e.g. Windows used to run on top of DOS? Or User Mode Linux on top of Linux?
An operating environment means you don't have to worry about stuff like device drivers. It can run inside a Docker container. (At most companies, try suggesting "let's run a brand-new OS that nobody has heard of", and you'll get an emphatic "no"... say "here's this app we want to run, it is packaged as a Docker container", and often nobody will even ask what is inside that Docker container, even if it contains your operating environment with the app running on top.)
You can start out using facilities of the host OS like the filesystem and networking stack. Later, if you want to, you can implement your own filesystem (create a 100GB file on the host filesystem, pretend that is a block device and your custom filesystem exists within it). Or your own networking stack (which can run in user space using facilities like Linux tun/tap drivers.)
An operating environment can always evolve into a standalone operating system at a later date.
Well, there is always a market for a new OS as they all are so complex and have so many baked in assumptions, that they cannot possibly do well on the variety of use cases that people need computing for.
My dream is simple OS that works kinda like DOS:
* Micro OS - Boot base from floppy drive if need be
* Let me edit a text file to left put in the pieces I wants instead of a hundred pieces I don't
* Includes BASIC language that build build a binary
* Includes text based GUI for a text editor
* Allow me to pick the drivers I want
I like concept of some Linux OS'es but the ecosystem has got some huge it takes an expert to truly understand it. For a server I really only need a network drivers, SSH client, and the server software. I maybe nostalgic but I miss the good old days.
It’s not like Multics or ITS or the Lisp machines or the IBM OSs were simple at all. In fact, they probably were more complicated. What you’re talking about are the toy OSs designed for extremely limited hardware for people who probably couldn’t tell the difference in the first place.
But if you want something simple, try out the base OpenBSD install. No magic, no complexity, just simplicity and elegance.
188 comments
[ 2.0 ms ] story [ 234 ms ] threadhttp://hal2020.com/2013/02/14/winfs-integratedunified-storag...
There's no sense storing videos, baby photos and "quarterly report new (2) backup newest 08.2019_john.xlsx" in a database.
I was looking forward to seeing it in action based on reports of what it was.
WinFS imposed all of those things and the associated overheads and much, much more every time you accessed anything.
Ever heard of Oracle Database Filesystem (DBFS)?
https://docs.oracle.com/en/database/oracle/oracle-database/1...
It uses soup unions to manage removable storage, and worked really well for the 512k memory available.
These days I'd recommend an embedded SQLite database for program configuration and data - somewhat standard, easily recoverable, backups are easy, uniform access from a program's API etc.
Lastly, I'd say the ultimate OS would have a bit more than this surely - minimal formally verified TCB, Arrow datastructures, Managed (but not GC'd) memory, native scripting language that blurs the distinction between users and developers, fully scalable auto-generative but skinnable UI, just to name a few.
I guess the trick with all of this is a suitable language which forms the OS base (think LISP OSes, Micropython, Squeak, AOS, NewtonOS), you get that correct and the rest can be layered on top.
Or AS/400, which is more or less a SQL database.
People keep on saying that, but how really true is it? Many operating systems come with bundled relational databases–e.g. most Linux distributions come with more than one relational database implementation bundled. Does that make Linux "more or less a SQL database"? How deeply integrated is DB2/400 into the OS/400 kernel (or equivalent term, such as "System Licensed Internal Code")? There is a paucity of public information about the actual nature or depth of this integration.
I think an OS with a truly deeply integrated SQL database would expose things like a table/view of all files in the filesystem, a view of running OS processes, etc. As far as I am aware, OS/400 (or "IBM i" as they are calling it nowadays), doesn't have any such features.
To add to this confusion, IBM ported DB/2 to OS/400 atop the native OS/400 data object model. You can use SQL but that just gets compiled to native operations against what are known as "physical files" (confusing name, yes) and "logical files". Physical files are fixed-length record files with data in them. Logical files are indexes atop physical files. A PC analog to this would be FoxPro, Dbase, Paradox, Alpha, etc.
Many OS/400 programs access these files using the native model; not SQL. Tandem and HP Non-Stop work in very similar ways.
[1] https://en.wikipedia.org/wiki/Single-level_store
I get the impression that S/38 and OS/400 have some really interesting concepts at the core of the OS, but it is questionable how well the higher levels layered on top leverage those concepts.
"Everything is an object" would be a lot more powerful if IBM let customers/ISVs define their own object types, when as far as I am aware they don't. Yet IBM will define dozens upon dozens of object types for all kinds of obscure requirements, many of which are no longer even relevant today [1][2].
[1] https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_74/...
[2] https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_74/...
The thinking at the time was that disk drives would become obsolete, and would be replaced entirely with some form of solid state memory.
What were they smoking? Bubble Memory?
Windows has it, though user-level tools are lacking. https://docs.microsoft.com/en-us/windows/win32/extensible-st...
Attempts were made to use it as a database, but stuff was always getting corrupted.
The original MacOS suffered badly from being a cram job into 128KB with no hard drive or MMU. No CPU dispatcher, no memory protection, brittle ResEdit and TextEdit. The trouble was, that architecture persisted for 17 years, long after the hardware improved.
The Mac grabbed most of the good UI aspects of the Lisa, but stripped out all the decent OS concepts. It worked to sell the first Macintoshes, but didn't scale out later, and by that time they'd killed off the Lisa.
This made a lot of sense for a system intended for high reliability transaction servers and nothing else. Banks loved Tandems.
https://www.ibm.com/support/knowledgecenter/en/ssw_aix_72/ge...
With that being said, my ultimate distro is Linux with a Desktop Environment that is less braindead than Gnome.
https://www.winehq.org/ https://www.darlinghq.org/
Nah man, sadly most of the really useful software is quite unstable under wine, if it even runs :(
I'm currently trying to figure out how to properly drive a 4K monitor with my laptop and not have it end up lagging. Supposedly hardware acceleration is enabled on Firefox, but even trying to play even a 1080p YouTube video drops frames. On Windows it is buttery smooth, so it's not a hardware issue. Don't even get me started about scaling...
(Currently using GNOME on Wayland)
https://www.phoronix.com/scan.php?page=news_item&px=Chrome-N...
https://bugzilla.mozilla.org/show_bug.cgi?id=1210726
https://wiki.archlinux.org/index.php/Hardware_video_accelera...
Windows sorta does that with the Registry but A) the registry isn't the filesystem, B) the registry is a filesystem on top of NTFS and C) the registry sucks.
If the Registry had the power of,say, PostgreSQL to back it up, it would suck a lot less. Of course, for large files, you should still be able to rely on more conventional filesystems as well as for backwards compat.
It has no schema. Imagine if it had some sort of schema, declaring what sub-keys and values are allowed under each key. Imagine if the schema was self-documenting, with each key/value declaration in the schema had an associated description explaining what it was for.
Imagine if it had richer data types. For example, a "link" type, in which a value actually has the name of another key. If you try to delete the target, either it doesn't let you, or it sets the value to some sort of null value. (Basically, something like foreign keys in a relational database.) And an index to quickly find "back-references" (show me everything that points to this key.)
The registry should have included a database of installed packages/applications, and every key should have been marked with what package/application owns it, along with some sort of indexing to make it quick to find everything a package/application owns. Application uninstalls would have been a lot cleaner, and issues with apps leaving behind junk in the registry avoided.
Transactions: This was added in Windows Vista. But it could have been there from the beginning.
OTOH, remembering the registry was originally implemented in Windows 3.1, which had minimum system requirements of a 286 with 1MB of memory, maybe my suggestions above just wouldn't have been feasible.
http://preserve.mactech.com/articles/mactech/Vol.09/09.11/Ne...
As that page describes, the "query" command (or its equivalent GUI) can be used to write filesystem queries, e.g.:
Really bad example here IMO. All OSes have the concept of opening by double-click. And OSX is not consistent about dragging to trash, eg, can you drag an open browser window to trash? If yes, what does that do? Does it delete the loaded web page from your computer? Or does it leave bits of it in various caches? Or does it delete the browser from your computer? Or does it delete the content from wherever it is a stored (an email in gmail)?
> The hierarchical organization of file systems manifests itself in nesting of directories (folders). On the other hand, directories are merely named views of a certain subset of files, selected according to some criteria. Thus, a directory can be thought of as a "database view," a named database query. It follows immediately that a file may appear in as many "directories" (views) as one wishes to. For example, one "folder" may show all files tagged as "sales reports", while another directory contains files modified within five days. Searching a file system and creating and populating a folder becomes therefore the same activity. Since saved views are database objects themselves, one can reference views within views if one so wishes. There is no required hierarchy however: one may create two views that refer to each other, or any other network of views that best suits the problem.
I built koios to solve this problem, but I didn't take the fuse approach because it seemed like a lot of extra work for the user, and puts databases first rather than files, which is un-UNIX.
Every single access of every single piece of data, for anything, will incur all of those overheads every time. There is no way to opt out or choose a different balance of guarantees or trade-offs without implementing them on top of the OS provided ones.
In a database, preparing a query compiles the access plan (ie all the above) on the server side. There's no reason it would be any less efficient than an OS call today.
Could in fact be more efficient, since you would probably be able to leverage set-based operations on the server (ie OS) side rather than iterating everything on the caller.
'On the server side' isn't magic. You don't get server side operations for free and these 'server side' operations would be occurring on the same computer.
Personally, I'd like to see version control and file compression integrated in my filesystem. I do not want my operating system to be distributed or rely on network access.
I also think a separation between administrative access to the full system and views of applications may be a good idea. It seems to me that the ideal OS would heavily restrict the file and network access of applications and there would be a separate administrative domain with full access but limited complexity. The operating system then manages the sharing of needed data between applications and network access that the user allows. I think having the system parse a standard configuration file format for applications would make sense in this model, but that configuration file can still just be text. Also having a sub-user permissions structure similar to the overall system users seems like a good idea to me. Many applications that users run are hostile and dealing with that requires major changes vs current desktop operating systems. Database vs. filesystem is a minor issue in comparison and personally I don't see the appeal of using a database.
It's not clear whether the author is arguing that all existing OS data structures should be replaced with a single database system, or if they simply want all OS data to be capable of being queried/updated via a standard database style interface.
The former is simply hubris ("I know the perfect data structure for EVERYTHING!") while the latter is pretty much implementable as an interface layer on top of an existing OS.
For more info check https://www.researchgate.net/publication/2364452_Data_Struct...
Scalars, list, tables(relations), tree/graph.
You can build generalized operations on them and it will work even if the internal structure (ie: list can be done on top of vectors or linked list or tables or tree) differ.
This is what I working on in my own little language (http://tablam.org). Certainly requiere specialize per structure and some stuff is hard to generalize but the idea hold.
The author mentions this removes the need for ids, but really it would just be hiding them in a non-serializable way.
not if you look at average consumer patterns (sadly)
- are composed of multiple processes; can include process management code
- can access hardware directly (for efficient arrangement, caching and retrieval of data)
- include support for multiple users, concurrent sessions
- stored procedures require compiler tooling and runtime (VM)
- can be platform-independent, with an abstraction layer connecting them to the underlying OS
It obviously wouldn't be worth the effort, but how many times faster would our computers be?
Shame google didn't use it instead of Android, they had to invent Go to get back some of the AOS advantages.
https://en.wikipedia.org/wiki/Bluebottle_OS https://www.research-collection.ethz.ch/handle/20.500.11850/...
I want linux with:
— Wayland
— KDE and only KDE
— Development SDK based on Qt (other toolkits strongly restricted, gtk and gnome gone wrong way)
— Packaging based on something like APK from android or like DMG (no repos by default, only for base system maybe, but other methods of installing not restricted).
Snaps and flatpack are trash because their goal is so called security and isolation, when normal human being need just convenience (hate repos on desktop).
Yes, it's macOS model, but with linux you can have different filesystems.
You could certainly make a distribution in which using anything but Qt would be very very difficult. But who would want to use such a thing? Besides you, of course.
Moreover, your comment isn’t a cogent response to the post. Did you even read it? He’s talking about a new paradigm of OS, something fundamentally different. What you’re talking about is the same old same old, but worse.
Yes, my bad.
> Did you even read it?
Finished now. Good idea!
> I don’t think very many people agree with you.
Is "people" limited only to linux fans and "libertarian" ones?
Most people on entire earth do not want (and don't have spare time for this) to always make any choices, and they shouldn't.
It's all about the user experience. System should be friend and still enable the user to increasingly get more out of the computer. It also needs to be extremely responsive.
AmigaOS got a lot right back in the 80s. BeOS (and now Haiku as spiritual successor) took a lot from that (system kits vs amiga's default set of shared libraries, and the concept of datatypes) and added some concepts of its own.
Meantime, the mainstream systems have only gone backwards.
My dream of a general purpose OS is definitely an open source microkernel, multiserver RTOS with capabilities and an user experience that builds on those two systems. It is important for it to be an RTOS, as unbounded response times would make the system fail as a general purpose system and as a personal computer system.
The closest existing system would be Genode. It currently only meets the technical side unfortunately, but it could meet the user experience requirements with some work.
Ignoring drivers for the moment, your typical desktop application does not have a need for a bounded response time. Desktop applications typically do not fail due to a dialog box being popped up a few milliseconds late.
The underlying drivers, on the other hand, do tend to need bounded-response time, but this is typically performed by the use of the interrupt handling subsystem, whether directly handled or split into upper/lower.
Even non-RTOSs should be able to provide real-time response to drivers in this way and the trade-offs taken by an RTOS scheduler are not always appropriate or optimal for a desktop, user-facing system.
No they won't fail but a responsive user interface is critical. A process running on an RTOS won't complete any faster (probably the opposite), but displaying some sort of acknowledgement that a key has been pressed or a button has been clicked without any human perceptible delay is really nice. I'd love for the user interface portion of an OS to run with real time constraints.
That's just the easy way to achieve hard realtime.
It is, fortunately, not the only way.
Not a few milliseconds, but a few dozen seconds might be a different story. While it might be hard to predict or measure an exact number, your deadline is however long it takes for the user to equate it to "forever" and kill the process / leave the website / uninstall the app / etc..
In general, being able to make these kind of promises means that the OS has to be pretty conservative about the code it allows to run, and a 'best effort' OS or some 'soft' RTOS is probably going to be faster or more efficient.
In terms of UI responsiveness, I'd bet that crappy software running on a 'hard' RTOS can still manage to feel sluggish, because the UI delays probably aren't actually in the OS itself.
I remember people who used the QNX demo disk on old machines describing how they could do big compiles in the background with the system still responsive to input. I'm guessing the parts interacting with the user had higher priority.
Real-time might help in another way: mitigating covert channels. Gotta know the timing of secret-handling code. Then, make the observed timing fixed-length or random. Don't always need the timing to do that, though. For instance, the separation kernels just need timing for the partitions with whole thing stopping regardless of where code is at.
One last benefit might be easier performance monitoring and diagnostics. If it's predictable, one might be able to assign a range or profile to it. Then, raise alarm if anything goes out of profile. Just speculating here: never built one.
A great many people work with audio or want to work with audio, and Windows and OSX are lightyears ahead of Linux in terms of ease of use and overall quality.
You'd think the PulseAudio fiasco would have improved the situation, but I believe the developers thought as you did, and so built a system that's woefully encumbered with latency. Jack improves the situation WRT latency, but it comes at the cost of an altogether brittle and broken integration with PA, and thus with all major Linux desktops.
Bounded response times may not be needed for your word processor, but there's plenty of folks out there who would be better served by it. Hell, I'd love a snappier editor on account of predictable response times to keypresses.
PA is in rather good shape now & it has capabilities its predecessors didn't. But it's very hard to get people to update their notions about something once it sets in.
No matter how much Microsoft tries, Windows will always be 'insecure, virus-ridden', macOS will always be super stable, beautiful, no matter how much Apple screws up & Linux will always be unusable on the desktop no matter how many decades I'll spend enjoying it on the desktop at home and at work.
Sadly, it seems a person's mind tends to naturally lean conservative and it takes a herculean effort to regularly update one's preconceived notions.
It's an exercise on self-delusion to pretend PA can replace ALSA as long as it has this blatant issue.
Patch sets like -ck simply don't help with this.
Try running cyclictest from rt-tests for a while to see the true horror of Linux behavior.
Or, in other words, the operating system is not general purpose, because it is not suitable for these uses.
Do note these uses do include the likes of audio. If there's no way to guarantee latency, then a spike will surely eventually cause an overrun, which will be perceived as an audio hiccup. Sometimes, this is just an annoyance. On a live performance or in pro audio work, it can be fatal.
A good microkernel will allow mixed criticality tasks to share the system. A lot of recent research in the topic has brought SeL4 to this point.
This sounds more like a complete version of the Fuchsia Operating System, which has both technical and is more of a user-facing OS. That probably has more of a chance of meeting these requirements at the rate that it is being developed.
I see it as
1) Thomas ptachech 2) idlewords you ass inferior trastard piece of skit
And never less 0) tryingq which is you.
Are you so ascetic? Don't tell us you got a text buddy on the side
For such a project you don't really want to just adopt existing ideas and implement them. I don't really understand why Google is doing Fuschia for this reason: architecturally it's nothing special. Is the GPLd Linux so bad? It took them this far.
If I were to do a new OS I'd explore ideas like all software running on a single language-level VM with a unified compiler (e.g. a JVM), a new take on software distribution ... maybe fully P2P, rethinking filesystems and the shell, making it radically simpler to administer than Linux, etc. You'd need to pick a whole lot of fights and make a lot of controversial decisions to justify such a lot of work: you'd need to be right about things other people are wrong about, not just once but multiple times. Then maybe if people see it working well they'd join you and port existing software, albeit, the porting process would inevitably involve some rewriting or even deeper changes, as if you can just run existing software and get all the benefits you probably didn't change anything important.
An operating environment means you don't have to worry about stuff like device drivers. It can run inside a Docker container. (At most companies, try suggesting "let's run a brand-new OS that nobody has heard of", and you'll get an emphatic "no"... say "here's this app we want to run, it is packaged as a Docker container", and often nobody will even ask what is inside that Docker container, even if it contains your operating environment with the app running on top.)
You can start out using facilities of the host OS like the filesystem and networking stack. Later, if you want to, you can implement your own filesystem (create a 100GB file on the host filesystem, pretend that is a block device and your custom filesystem exists within it). Or your own networking stack (which can run in user space using facilities like Linux tun/tap drivers.)
An operating environment can always evolve into a standalone operating system at a later date.
I like concept of some Linux OS'es but the ecosystem has got some huge it takes an expert to truly understand it. For a server I really only need a network drivers, SSH client, and the server software. I maybe nostalgic but I miss the good old days.
But if you want something simple, try out the base OpenBSD install. No magic, no complexity, just simplicity and elegance.