Ask HN: Has anyone ever tried building an OS that doesn't use files?
Since the beginning of computer time, all computer systems have been based on the idea of a "computer file". OSX, Linux, Windows, and every other OS has been based on the system of files and folders and filenames and crap like that. I'm not too versed in computer history. Has anyone ever tried making a computer system that does not use files?
In my opinion, referring to the data that computers deal with as "files" made sense in 1954 when every large company had a "paper file system", but in 2014, I think its time to come up with a new system that better mirrors how we use computers.
I'm working on a project I'm calling "Library Transfer Protocol", which is aiming to replace the concept of "file" and replacing it with 'Library item'. Basically, in 2014 computer usage more closely mirrors the workflow of an author (revisions, publishing, etc), rather than an employee filling up a file cabinet for internal use (thank to the facebook and the like)
Before I get too into this project, I'd like to know if anyone has ever tried doing something like this before.
66 comments
[ 5.6 ms ] story [ 133 ms ] threadirmin, http://openmirage.org/blog/introducing-irmin
plan9, https://www.cs.unm.edu/~fastos/05meeting/PLAN9NOTDEADYET.pdf
First, look into history more--there are several non-hierarchical (read: flat) file systems out there.
Second, while the workflow might mirror authoring more closely (which I think is horsehit, but that's neither here nor there) the artifacts of that process are what matter. Existing notions of a "file" map very cleanly onto the storage and organization of such artifacts.
There is an argument to be made for having better querying capabilities or permissions or whatever, but what is to be gained from throwing a commonly-accepted idiom away?
For instance if we throw away certain programming features from a language, we can gain reliability and security which threaten even the integrity of code that doesn't itself use those features.
The hierarchical file system as you listed it really only started to come into its own in the mid 60's. With LISP machines at MIT and Multinix at AT&T.
Storing data in files as you call it is old, and well known solution to this problem. Because finding a node on a tree is simple, and this is how file systems tend to work. Because thinking of objects, as subsets of various super classes of objects is easy for people to understand, when you don't explain it in those words.
The reason very old OS's didn't store things like this, is because there weren't much permanent storage. Actually MTS uses what are roughly files but uses a dot notation to seperation files. Which will look similar to usenet
Where data is your current record.:.:.:
I support moving to a more revision, publish, etc. structure. But moving away from the tried and true hierarchical model will be difficult. Even an object based file system will develop a hierarchy of inheritance.
There was some time after its creation (as I remember hearing, at least) a "multi-segment file" abstraction created for data sets that were > 1 MiB, something of a kludge.
http://en.wikipedia.org/wiki/WinFS
An interesting single-level store was discussed by Robert Strandh in his 2004 proposal for a Lisp operating system, Gracle. I can't find the original paper on the web anymore but some pertinent excerpts are in https://github.com/jon-jacky/Piety/blob/master/doc/gracle_ex.... Strandh referenced another experimental OS with a single-level store called EROS. I see he has a more recent LispOS at https://github.com/robert-strandh/LispOS.
Makes searching and organizing a pain in the ass.
How did you come to this conclusion about patterns of usage? I'd think the typical user/consumer would more likely have 1000 mp3 files rather than 1000 personally authored Microsoft Word documents (or Photoshop PSD files, etc.)
What about another common usage such as digital camera photos? The digital camera (or iPhone) has jpg "files". How would the user mentally translate the "files" living on a FAT32 flash card and copy them to your "Library Items" storage system? Do they keep 2 mental models of storage paradigms in their head? If your proposal includes a driver/wrapper for hiding the FAT32 file inside the concept of "library item", it seems like you're just renaming "files" to "library items". It's more a shift in terminology rather than shift in paradigm as a sibling comment already noted.
The filesystems in an operating system (NTFS, ext3, etc) are already implemented as special purpose databases. The "rows" are file id entries and they each point to the "blobs" which are the file contents. Whatever you propose to build has to reimplement this underlying "database" as well. Whether you call the rows of that database "files" or "library items" or "objects" or "documents", it isn't going to revolutionize the approach.
Instead of the user supplying a filename/path for new content, they submit arbitrary metadata. When retrieving content, they query via a language called LQL (Library Query Language).
Basically the system works a lot like ID3 tags for mp3 files. 'Arbitrary metadata' is a lot like the fields for 'title', 'date', 'album', etc except they are arbitrary (you can use any set of key/value for metadata items)
Libraries can connect to other libraries. A 'Library' is much like an email inbox, except its one user per machine instead of multiple users per machine. Each library is 'addressed' via a domain name. All communication between libraries is done through HTTPS.
Traditional filesystems use a permission system that is very archaic. In 1960 it was common for there to be multiple users on one machine, but in 2014, everyone has their own machine. File permissions are useless, since I'm th one person who ever uses this computer. "Permissions" in the 2014 sense is which of my facebook friends gets to see these photos.
So, MongoDB with a file URL? (Hint: that's you could implement the MVP of this, and if you use a URL you can even reference user files they don't store locally)
And file permissions are dead? Because nobody has kids that use the same desktop they do?
This is (from a technical standpoint) the silliest goddamn thing I've ever heard.
From a product standpoint, you could probably pitch and get a few M. Why the fuck not.
What you said about a glorified metadata search is right on.
Some people share devices, but most people we share with use other devices.
I'm glad you think its silly because all great things have haters (bitcoin, justin bieber, etc.)
Stupid things have haters too. Some people "hating" your idea doesn't make it good. Doesn't make it bad either.
I don't know what windows does these days, but OSX has had tags and metadata for "files" for a while. It's even easy to add the info, and search for it. I never bother though because, for me, it doesn't solve any problems or add any value.
"It's responsible for handling user metadata such as tags, rating and comments. It also handles indexing and searching for files, emails, contacts, and so on"
Cross-device research for semantic desktop: http://www.dime-project.eu/en/home/dime/project/solution/con...
Many modern filesystems are littered with the reeking remains of attempts at supporting metadata (for example, NTFS), most of which nobody cares about and which just add implementation complexity.
If you want to pitch a more useful and more abstract version of what you're describing ("how can we present searching and accessing a metadata forest backed by traditional hierarchical file stores") then by all means I'll be friendlier but right now you're coming across as a crank ignorant of the history of the ideas you're decrying.
http://en.wikipedia.org/wiki/IBM_System_i
Unlike the "everything is a file" feature of Unix and its derivatives, on OS/400 everything is an object (with built-in persistence and garbage collection).
http://en.wikipedia.org/wiki/Newton_%28platform%29#Operating...
It didn't have files, it had database records instead. In general each app used its own database, but it was possible to read databases from other apps too.
begins trailing off... I wonder if I still have any old PalmOS app code laying around on an old hard drive somewhere. Would be fun to have a look at that. I think that was the last time I did C++.
Don't try to do it by applications registering types they can open - this never succeeds, there are simply too many file types in the world.
Also think about how to send data to someone else.
And finally think about how to integrate with existing devices that still use files.
Its a lot like the never-finished WinFS from Microsoft.
Funnily, modern file systems (extfs3/4, NFTS, HPFS+?) all support extended attributes in some form or another. However, they are currently only rarely used: Mostly for the "this file was downloaded from the internet, do you really want to open it" flag. I wish more programs would use them to store interesting metadata, but it's basically a chicken and egg thing now.
Windows and GNOME also have concepts where you can have calculatable attributes - you have a little library that looks up metadata in a database or parses it from the file, and then serves it as an additional attribute on the file (visible in the file properties tab). You can see it e.g. on mp3s or word documents in windows. However, it doesn't seem to be widely used either, and I wouldn't be surprised if that function has been gutted out of GNOME lately.
For anyone interested in these, they're called Alternate Data Streams in Windows. When you download a file from the Internet, an ADS is created, called "Zone:Identifier", which contains a ZoneID indicating where it came from.
I believe Windows 2000's Explorer allowed the user to add arbitrary tags to any file, and that those were stored in an ADS. For some reason, that was removed in, I think, Windows Vista.
At least one virus stored itself using Alternate Data Streams, which I imagine is related to why they've been more or less downplayed.
http://www.plan9.bell-labs.com/wiki/plan9/using_pq/
What does this mean? Are you defining "operating system" loosely enough that you could also count the JVM, for example?
The Wikipedia page for Operating Systems says "An operating system (OS) is software that manages computer hardware and software resources and provides common services for computer programs", and by that definition, the JVM would count as one.
Though most users also expect some GUI/Desktop from an OS, and that's what browsers offer, but not the JVM. (The JVM allows programs to have a GUI, but it doesn't have a GUI itself, for example for launching other programs).
That and you can actually boot from Emacs: http://www.informatimago.com/linux/emacs-on-user-mode-linux....
Really, by your definition, any piece of software with a plugin-based architecture is an operating system. There's certainly a trend we're witnessing where, for the end user, the browser is displacing the underlying operating system, and turning the latter into an environment for hosting the former. But to call the browser an OS is a huge stretch.
But security and sandbox restrictions are very important elements of an OS, don't you think?
Really, what you seem to want is a file system with built-in version control and network sharing? Git and Mercurial are already virtual file systems of sorts, I guess.
If providing support for application-level user-friendly abstractions is what you want to do, then I would suggest studying applications with UI abstractions that you admire and judging your OS storage layer by how well it supports application development.
Please define how a "library item" is different from a "file".
Is it made of bytes that can be read into a buffer and accessed?
(If not, how can an H.264 video or MP3 object exist as a library item and be processed?)
Do you not have spaces which assign names to library items?
The Hyper-Text Transfer Protocol has already replaced the concept of "file" with "resource". A URI doesn't necessarily name a file.
The user constructs a query then sends it to a server, the server returns a list of items that match the query. For instance this may be a query:
INCLUDING artist contains "Pink Floyd"
and the results will look like this:
[ {"artist": "Pink Floyd", "title": "Another Brick In the Wall", "mimetype": "audio/mpeg", "album": "The Wall", "url": http://drive.google.com/blahblah.mp3" etc }, {"artist": "Pink Floyd", "title": "Money", "mimetype": "audio/flac", "album": "The Wall", "torrent_url": http://drive.google.com/blahblah.torrent" etc. }, {"artist": "Pink Floyd", "mimetype": "image/jpeg", "album": "The Wall", "url": http://drive.google.com/albumcover.jpg", "purpose": "album artwork" etc.} ]
You can add more to the query to filter out the content you don't want. The query language is much like a SQL WHERE clause. The query language is meant to be super simple and something your grandma could figure out.
The app then can retrieve the actual file from the url. The end user has no idea about the underlying file crap.
"NetKernel can be considered a unification of the Web and Unix implemented as a software operating system running on a microkernel within a single computer."
https://en.wikipedia.org/wiki/NetKernel