Ask HN: What's the next evolution of the GUI OS?
What if we remove windows completely? Think about it - when you open a program, you don't actually need a separate window. What you need is functionality! So what if we remove windows and keep only the functionality. For example, you open a .doc file, but there is no separate window. You stay in the same "Desktop", but now you have an open file and functions to work with this file. And if you need some other functions, you open another program - and other functions appear on the desktop. So now you have two different programs working as one! And you don't need to arrange different programs and files on the screen. Because now you can have different programs and files working simultaneously as one.
I think that will be an incredible experience! And it will tremendously increase productivity and improve the workflow. If you're interested or have a comment, let's discuss below.
69 comments
[ 5.2 ms ] story [ 135 ms ] threadWhen I open two .doc files, I can only have two separate windows or tabs. But I don't need those windows or tabs. What I actually need is just two open files on the screen. And some Word functions. From UI perspective, it can be a "function bar" on top of those two files.
If I want to open an image file and Photoshop, currently I can only have a separate Photoshop window. What I actually need is just an open image file on the screen. And some Photoshop functions - a "function bar" on top of that file.
So the visual difference is that you now have 3 different open files and some "function bars" - all in one workspace. And you can arrange or scale your files in any way you want. And you can arrange "function bars" on the screen or hide them.
But, whatever you do, you need some kind of borders to understand where one document ends and the other begins, and you need some controls to change the view of each individual document (such as minimizing or maximizing it) and you need a button of some kind to close the document. Then, if we have two documents and floating bars that you can put wherever you want on the screen, it will be completely impossible to tell which bar controls which document (or displays the state of which document), so we should probably put the function bar somewhere in the document container. And, hey presto, we've reinvented the Window, because it turns out to be a really really useful and natural abstraction.
Many websites break down when you scale them beyond. For example, news.ycombinator.com goes out of screen at >200% on my screen.
My terminal default font size is 20, but I often scale it above that when lines output is short enough for a period of time.
I love big output without fluff like task bars and clocks and other clutter.
What time is it? I don't know, I'll check my WRIST WATCH! :-)
Why would you want this? Well, apparently, everyone these days likes watching movies and such on vertical screens, so that's the way the industry needs to move. I don't get it myself, but then again I'm not someone who watches TikTok videos.
Large laptop screens (>15") can be sufficient here, but two external monitors are priceless.
or at least the Acme editor, which is also available for Mac and various Unix systems like Linux, FreeBSD etc.
https://github.com/9fans/plan9port
It became unbearably obtuse to use on a laptop, and I didn’t like any of the workarounds, so I eventually upgraded my “terminal” to Linux and a custom LXDE setup.
e.g. You have a TSV file which you might want to align visually and edit multiple fields simultaneously, so you enable `tsv-mode', and then you enable `multiple-cursors` and you have the desired editing functionality on that file.
But to answer your first question: I genuinely envision my ideal future desktop as being some smart-glasses kind of thing, where I can type by thinking. Right now, what my mind wants and what my hands want are often aligned, but my mind wanders. If I can type at the speed that I think, I have the impression that this will greatly increase my overall focus and mind-discipline.
(I can also envision employers forcing their workers to install intrusive mind-type monitoring apps, to ensure that they're actually working and not just drooling into space, but that's perhaps a separate discussion.)
You should be able to tell the AI engine something like "as soon as the weather forecast predicts snow, order a snow blower from Amazon". The AI can only do this if it has access to weather data and an API from Amazon it can control to get you the snow blower. It should also be able to check your calendar, because you can't accept the delivery if you are on vacation.
Apps are silos that prevents the AI from doing these things. At best the AI could use the equivalent of screen scraping to use existing apps, but eventually all apps should be replaced by APIs that just extend the capabilities of the AI engine.
And what about GUIs? I think they will be pretty irrelevant. The AI engine should be able to render your data any way you want. Think Dall-E. There will certainly be designs to chose from, but you should be able to customize them any way you want ("remove that italic button, I never need it").
>At best the AI could use the equivalent of screen scraping to use existing apps, but eventually all apps should be replaced by APIs that just extend the capabilities of the AI engine.
Why would companies want to expose APIs in this manner? Instead, they can force you to access them only through their app, and that way they can force you to sit through advertising, and otherwise control how you see the information and how you're allowed to interact with it. Open APIs cut them out of a lot of profits.
>You should be able to tell the AI engine something like "as soon as the weather forecast predicts snow, order a snow blower from Amazon".
This is crazy: this way, someone could just have their AI do these tasks in an automated way, without paying some company for the privilege.
Instead, what we need is an app that's the only way to access these APIs, and the app will give you ways to set up automation, but only in the ways the company wants you to, and poorly. Then they can charge you a monthly subscription fee for access to this automation app, and also embed annoying advertising in it.
They don't. But why would record companies want to license their music to streaming services instead of selling CDs? If the AI is there, and it offers a superior experience to users, the companies that don't support it will disappear.
> this way, someone could just have their AI do these tasks in an automated way, without paying some company for the privilege.
You're assuming that APIs need to be free, but that doesn't need to be the case. Companies that provide data will need to charge for that, just like record companies charge Spotify when you stream music.
Companies that provide a service (Amazon, Uber etc) won't be happy that they lose marketing opportunities and brand recognition, but again, if the user experience is superior to the old app ways, those companies that don't adapt will die.
For that to happen, the AI needs to already be working. If the companies proactively prevent this from happening, then users won't have any basis for comparison. Closed APIs will prevent them from having the ability to program their own AI to do these things.
>You're assuming that APIs need to be free, but that doesn't need to be the case. Companies that provide data will need to charge for that, just like record companies charge Spotify when you stream music.
Sure, companies could require you to have a subscription to access the API, but they can make more money by controlling the experience instead of just giving you (paid) access to the API.
>Companies that provide a service (Amazon, Uber etc) won't be happy that they lose marketing opportunities and brand recognition,
Yes, this is exactly my point.
>but again, if the user experience is superior to the old app ways, those companies that don't adapt will die.
Users won't easily go backwards to proprietary access when they're used to (more) open access to something, but this isn't the case in this scenario: the users haven't had easy access to APIs before in a way that a regular person can just, for instance, give a voice command to their system saying "when the weather is like this, do these things with these various services", so they don't know what they're missing. The services already control the experience, and will want to keep it that way.
I don't think that the AI will be primarily developed to replace apps. I would rather expect Alexa or Google Assistant or Siri to start supporting arbitrary APIs.
> Closed APIs will prevent them from having the ability to program their own AI to do these things.
I would expect that a company like Google is able to find partners, but if not, there's always screen scraping.
> but they can make more money by controlling the experience instead of just giving you (paid) access to the API.
Not if people stop using the controlled experience. It just takes one competitor to offer the API.
They can shut that down with laws banning the practice. It might not stop someone working on a personal project, but it would prevent any significant company from offering any kind of paid product or service using screen-scraping. In fact, it might already be forbidden just by ToS.
If you're a company, that's not a bug, that's a feature. Data is your asset and releasing it for free is almost always a net loss.
Actually one more way towards few companies controlling the internet. Because having an agreement with one supplier is more easy then contracts with many.
There is absolutely nothing in the above declaration that requires an AI.
The Affinity Suite has a nifty feature where, if you own the other packages, you seamlessly can edit a photo from Photo in Publisher for instance. Or a design in Designer from Photo etc. nothing loads, it just works.
For example in Windows you can drag and drop file(s) between windows to move those file(s) between directories. In my application you can achieve all the same interaction which may move those file(s) to a different directory, different personal device, or different user. The experience is the same whether the application is running on Linux, Windows, or OSX.
In this application when you double click a file, in the browser to execute it, the OS default application associated with that file type opens just like in the native OS. If that file is on another personal device or another user it is copied to the local device for execution.
In order to minimize clutter I have a button to minimize all windows to a tray, but that is hardly original. Soon I will start working on a terminal emulator (also displaying in the browser) for remote terminal access to personal devices.
I know many people don’t like interfaces written in JavaScript but it is portable to things like Electron and Tauri. The test automation executes in less than 10 seconds on just the local device or just under 2 minutes for full peer to peer testing. Because this is all JavaScript there is a bunch of low level concerns I don’t have to reinvent, like accessibility, but the socket based decentralization does require new transmission handling and some original security work.
My WFH environment I use every day has three screens stitched together for 7680x1440. On the left third is my primary remote desktop app that connects me to work with a window for my old desktop PC. The middle third has 320pixels reserved on the right for context-sensitve notes, then splits the rest in half with the left half being my timesheeting/ticketing app and the right half being a decent plain text editor as a scratch pad / diary combo. On top of that I have a set of tabbed windows (using Groupy) from various apps like a file manager, browser, Slack, email, or command prompt. The right third has the left 320 pixels reserved for a music player (Winamp), VOIP softphone, and clipboard manager. The rest of that portion is another set of tabbed windows usually at least including a second remote desktop session to my VDI and another browser window. My browser and command prompt are tabbed. My file manager is dual-pane and tabbed with a preview pane.
What you're describing sounds like what was tried and failed with Windows 8.
Longer term, I think voice + AI assistants can help with this. In programming speak, our UX will become more declarative and less imperative. Combined with AR/VR interfaces, this will see a big improvement in productivity.
This idea of a "universal container" as strong similarities to Apple/IBMs "project pink" and OpenDoc [1] in the 90s that went on to become Taligent [2] that went on to become... not very much. There was also Microsoft's roughly concurrent efforts with OLE [3] and Windows NT Cairo - OLE was a stab at what might be described as componentising application functionality at the UI level, and Cairo had (iirc) the ambitious goal of replacing (augmenting?) the windows file system with a semantic data store with OLE/ActiveX as the UI layer.
OLE did kind of work, and actually made it into products like Word and Excel - though whether that was a good thing is debatable. The rest of it basically got canned, probably because it was too difficult to implement and/or sell. But I'm sure the influence lives on to some extent. I'm not aware of any serious attempts to go down this route again.
I still have some OpenDoc SDK beta CDs somewhere. It might be interesting to get them out and have a look.
Personally I think that "application" as a discrete concept provides a useful conceptual boundary [4] - although I will admit that this may just be that my brain is by now hard-wired for it.
[1] https://en.wikipedia.org/wiki/OpenDoc
[2] https://en.wikipedia.org/wiki/Taligent
[3] https://en.wikipedia.org/wiki/Object_Linking_and_Embedding
[4] Both for people using the application and for the grubby activity of selling it to them. Got to have a "thing" to sell.
Fascinating responses thus far.
We're in a terrible state, some odd and ugly place between native and online, where even native apps are picking the worst from online and online is denied access from the best of native.
I want an operating system GUI that dictates the available widgets to the program, and hands me control of their presentation, I don't want applications, online nor native to be able to dictate their own look, only for them to present their available functionality to the OS, and have the OS compose the screens for me, probably guided by my predefined preferences, sort of how HTML could have been if CSS hadn't ruined the day, sort of how native winforms programs used to be done.
A concrete example is things that can save a file, I don't want the program to dictate the gui for this, I want it to keep the OS updated "I have these documents open which have been changed since last save" or "There are these things which can be printed", and have the OS dictate GUI for activating that functionality.. It used to LOOK that way, for a lot of things, you'd go to the file menubar, and there you would open/save/print your file, this was easy to learn and very consistent..
Currently, the most consistent interface experience on my computer is bash in xterm, in elinks, websites look pretty consistent, if they're well designed.
But if I open firefox, chrome, gimp and kate, nothing looks or feels the same.. Then I open blender and a fifth kind of interface appears. No thanks.
So I sort of agree with you that it'd be nice if we did away with the window metaphor to some degree.. But I was also no fan of the way Mac had the "god bar" at top (which I believe Ubuntu did too) terrible idea..
For now, the least crappy desktop experience for me is i3, but it does not change anything fundamental, it's still program windows, free to do whatever they want.
But, for some things, the most futuristic version of the environment is the one that does not need a screen.. The tasks that are silently done for you without needing you to transfer any significant amount of information between you and the machine, or being able to do so on another level than a screen (brain machine interfaces and whatnot)
I think the "dynamic-ism" of many interfaces has gone too far, therefore losing the predicability. Some interfaces try to guess what I am supposed to do next and it is correct maybe 60% of the cases.
But even when it is correct it actually slows me down because I am constantly second-guessing the interface and double-checking that what I am about to press is the one I actually wanted.
I wish I could rely much more on muscle memory and that things re-open where I left them. Like a window from a folder _always_ opening exactly where it was when I closed it previously. Or the homekit buttons on my home iOS pull-down screen not rearranging themselves randomly.
Edit: Another irritating example of this "trying to anticipate but keeps hampering" behaviour is the Outlook mobile app. When moving an email into a folder it will suggest three recently used folders and after a while re-arrange them after 3-4-5-something mails have been moved in a row.
I want there to be places for things, a fairly rigid system so I can depend on things being in their place, not having to constantly "try to understand what's happening now"
My browser can register that it can also display files of those encodings, and so I could effortlessly open a "viewer" in context of my text-editor, have it be my browser, and then when I edit the text-file, it is automatically shared (maybe even accessing same memory by pointer if I'm living dangerously) with the browser so I get live preview of my html.
The preview functionality in this case is not something that's explicitely supported by the text-editor, nor the browser.. It's just that since my OS already knows that the editor provides a source of text, and that the browser can view a source of text, then I can chose to open a browser perspective into the editor-source..
It might be that the operating system provides a "save file to disk" source, that I can then use to save the file from any program that provides a source of data.. It'd work the same way, instead of opening a browser as a perspective into my editor-text, I'd open a file destination into it, or I could open a LZ77 compression destination into it, which then opens a file destination, and my file is automatically compressed and saved, no program needs anything but tell the os what it can read and write..
When I was developing Palapeli (a puzzle game that ships as part of KDE Games), I did a KParts version of it for fun in a few lines of code. That made it so that if you had a puzzle file stored in a folder, you could navigate to it in Konqueror and the game view would come up inside of Konqueror.
Back in the day, I dreamed of a fully document-oriented computing experience. Sadly the world has settled on the service-oriented experience of today, largely controlled by a small number of gigantic fiefdoms.
The GUI would need a similarly smart approach and given an OO persistance layer I'd say that applications would just be objects that composed other objects - the document object would be your wordprocessor and it would be good at arranging and presenting other types of objects into 2d representations at A4 size.
The web browser would presumably just be a superset of the document object or might not even be needed at all as a web page is just another kind of document.
You'd still need a "fresh sheet of paper" concept but also a history so you could go back and search for what you were looking at/working on previously. Also different contexts - like all the stuff you need for one bit of work and another context for some other kind or play/work. I don't think windows are the worst way of doing this and I'm certain that you'll end up wanting to look at different things side by side or whatever and that will become windows by another name but there might be subtle improvements.