I do wish that apps could compose like command line tools. It'd be nice if you could apply a texture in Blender and then if you wanted to tweak it launch a window from Photoshop to edit it. Not launch Photoshop the whole app, just the the drawing window.
Not sure if it's still true, but in the glory days the DTP industry was known for stitching together different applications this way using AppleScripts
The reason this doesn't work the way the author wishes:
> The idea of co-designing language and UI
Is because commingling presentation, use cases, and biz logic is highly coupled.
UNIX files have just open, close, read, write, and seek in the simplest form.
The UNIX 'everything is a file' is really just a form of dependency inversion.
I am old enough to remember when the visual languages and UML were supposed to replace programmers with architects with GUIs.
But it ignored that commingling implementation details produced systems that are tightly coupled with implementation details and is difficult to work with.
It seems that these principles are ignored and rediscovered over and over again.
Clean, hexagonal, onion, and ports and adapters architectures being some of the most recent versions.
The UI should depend on the core system and that core should be ignorant of the UI as much as possible.
Adobe has no real reason to expose PS functions that way, and obviously I am talking about ideals. But it is just simple reality that when you tightly couple to implementation details you paint yourself into a corner.
This seems like a great idea! It was in fact a major focus on effort by basically everyone from about 1988 until 2004... This was the idea behind Microsoft's COM, IBMs SOM, smalltalk, Objective C and Openstep, many different projects at apple, and many more places.
It failed for reasons that are obvious in retrospect. Creating a system where you can embed one app in another requires everyone to agree the interface standard. But the interface is extremely complicated! You have to pass arbitrary data between two applications, and do so quickly. You also have to decide on how much the apps are isolated from each other (getting it wrong creates either massive security failures, or performance that is way too low). You also have to deal with such issues as how to synch the user interfaces!
So any such system is going to be extremely complicated, and getting everyone to agree on standards is almost impossible. What standards do get created are so complicated that they become unusable except for very stereotyped use cases. The other alternative is when a single company forces a standard down everyone's throat. But that creates the problem of vendor lock in, so people tend to reject that!
There are in fact a couple of notable successes. The most obvious is the digital sound market, which btw is a really interesting software market because there is huge competition. There are 4 major DAWs (digital audio workstations), and a number of still used minor DAWs, plus lots of companies that make synths, effects, tools you name it. And they all get sold! A huge amount of this software is in the form of "plugins" apps that run in the DAW as a container. This works because the interaction between plugins is a) similiar to unix pipelines (it forms a directed acyclic graph), so the interation is relatively controlled, and b) there are two standard data interchange methods. One is raw sound data in the form of 32 bit 48 kbs PCM, and the other is midi (a form from the early 80s for controlling synths). The user interface problem was solved in the easiest way... there is no user interface specification, or library, and every app has to roll its own interface (which it does by just using bitmaps... you end up with the most horrribly skeuomorphic designs you've ever seen!). There are actually 3 different standards for interfacing with the DAW, but one VST is dominant. Moreover, because of the simplicity of the data exchange format, the interfaces are actually quite simple to use, if you know about the weird world of audio programming.
The other success story is Microsoft Office. Office is based on MS's tech for creating components, known as com. When you a draw a graph in word or excel, you're actually using a different little app that draws graphs... and you can embed that in your own (windows) app. But COM is extremely complicated, and com components are almost used in apps specifically designed to be part of Office. There's a fairly big ecosystem of these apps (or there was) but its very tightly controlled and integrated by Microsoft.
You don't need everyone to agree on an interface if you approach things slightly differently. Like Unix shell scripts and pipes, each program can have its own particular inputs and outputs, as long as they're useful and documented... people will find ways to stitch them together.
This was the approach taken by Arexx. I've never used or seen IBM's Rexx outside of the Amiga version, Arexx. While a PC version of Rexx existed, it's obvious why it didn't take off... IBM wanted exorbitant fees per end user, and even higher developer fees. As a result it had no users, and with no users, no software bothered to supported it. Most PC developers likely had no idea it was even a thing.
It was different on the Amiga. Commodore somehow got the rights to including it in their OS, no additional licensing required. As such, there was a ready user base, and thus many apps had Arexx support. You could query data from one app, massage it in the scripting language, then pass it off to another app.
It was a well-loved feature allowing all sorts of automation that wasn't really available anywhere else.
you still need to agree on some things... For instance even if you're passing data through text, you have to agree on the actual encoding (Unix uses ascii, but there are lots of different text encodings!). You also have to agree on the meta-properties of the communication. How do programs find other programs to communicate with? How do they actually pass data to each other? How do they maintain transactional integrity (i.e. a request sometimes needs to be atomic, either happening or not happening, and the requesting program needs to know if a request happened!).
Apple's OpenDoc [1] was another attempt to break out of the silos of monolithic apps each completely responsible for their own document formats.
It died.
I don't think there are any real technical of programming language challenges to making this work. It's a problem of economics and sociology.
Economics:
If your doc/app/thing is composed of a multitude of widgets developed by different developers, who pays for those widgets to be developed? If you have to buy them, does that mean you get a doc that you can't send to anyone else unless they have they've paid for the same widgets?
(This problems exists today in the world of music production where there is a thriving plug-in model of third-party VSTs which are wonderful for extensibility but do make it quite hard to share projects with people.)
If that's not the business model... what is? I don't think anyone ever figured out a good way to make money making software like this outside of limited domains.
Sociology:
Conway's law [2] isn't a joke. It's a real observation that for different software components to interoperate, the teams making them need to communicate at least enough to create protocols.
Defining software protocols that are expressive enough to be useful but simple and stable enough to be widely used by completely independent competing organizations is really hard.
Imagine you had to design an API for composable UI widgets. How would you handle text rendering? RTL? Theming? Colors? Layout? It's not intractable, but it's not far from it.
Both:
Then you have to consider that if you have competing organizations trying to make money selling widgets, they don't want their competitors widgets to be successful. They are incentivized to undermine compositionality whenever they can (Microsoft's old "Embrace, extend, and extinguish" [3]).
Also, a fundamental tenet of usability is that the interface as a whole should be consistent and hang together. Users don't want apps that are a patchwork of jarring widgets each with its own style and convention. If you could get a platform like this off the ground, users would still like converge onto a small number of companies that made large sets of widget suites that are designed to compose together harmoniously. The usability benefits of doing so would work like network effects to lead towards platform consolidation.
Software people just dramatically underestimate how hard composability actually is in practice. It doesn't matter if you can shuffle bits between two pieces of code. That's the easy part. The hard parts are all in protocol design, usability, competition, etc.
Visual Basic succeeded at this somewhat. There was a market for widgets. You paid for the widgets and your users paid you for the composed application (the users were not free to recompose it however). But that was in the days when people paid for software
Nowadays people don't pay but it doesn't mean you can't compose things from open source stuff like material UI.
I think that Bracha's point is that it's always been hard for a user to compose apps and make them consistent etc(although VB was much easier than web stuff). But now the AI will compose them so the promise of VB etc could be fully realised. Power to the people.
It really was amazing how large the market was for paid Visual Basic VBX/OCX custom controls (widgets) in the late 90s. This article estimates there were more than 100 companies selling them by 1997: https://www.devx.com/tip-bank/23792/
I do think that the problem with Bracha saying the AI will compose things is that as Big Dave Thomas once said: "It's important to be able to debug at the level of the abstraction". I'm not sure that even with automatically tested code and even some proof we will ever get away from the need to debug.
Dave Thomas was talking about model generated code and models were a much more precise starting point than the prose we input to an AI.
About a year ago I read the OpenDoc specifications [1] (which for some reason are not linked to from the Wikipedia entry on OpenDoc). What struck me was how much of the complexity was to allow different vendors to provide differing amounts of services for comparable components - all sorts of fallback mechanisms for rendering the product of one spreadsheet component if you have a less complete spreadsheet component, for example.
I will add that, even in a business setting, open source package repositories are pretty good for sharing code and a closed-source marketplace would add a lot of friction. When people at different companies want to collaborate on making improvements to a library, do you really want to negotiate a contract?
Seems like this is coming from someone in a silo of his own, nostalgic and out of touch with industry. I had a Newton and I don’t miss it.
In particular, JavaScript has evolved into a much nicer language than it started out, even more so if you add TypeScript. Yes, there are unfixable gotchas, but they’re well-documented and they’re far less important than some critics think. They rarely get in the way. There are tools that could be better, but they’ve improved a lot, and keep improving.
Also, embedding one app in another visually is a poor fit for mobile devices. There’s only so much screen to work with. Share buttons using common data formats mostly do the job. On the web, the best way of sharing something widely is with a URL. We have iframes but rarely use them.
Newton might be more interesting for the R&D and pantheon of technologies around it (Dylan, etc) than for the actual implementation, which, yeah, was cool for its time but we can do much better now.
I'm trying to remember the name of a dedicated document-processing system; IIRC it had 256k of RAM and it would save its state by dumping the RAM to a 256k formatted floppy. It was also quite integrated, but didn't run arbitrary apps.
I liked his book, The Humane Computer, when I read it in 1999. I don’t know how well it has aged. It’s still on my shelf, so perhaps I will try to reread it.
Word originally saved this way too - just dumping memory to disk. But also by just appending the diff to the end of the file, letting writes be faster.
Unix CLI philosophy, COM, ActiveX, Desktop Widgets, AppleScript, IFRAME's, The microservices movement, iOS Shortcuts, IFTTT, WebComponents, audio plugins, etc -- consumer clients and backend services have continuously explored composability as an engineering approach. Enterprise products often drown in bespoke/internal approaches to it.
It's an obvious idea, sometimes productive, but tends to accumulate incoherence, impedance, and security issues over time so (with exceptions) any given implementation tends to burn itself out eventually.
There are living examples now, there were many examples in the OP's 2005 world, and there will be more in the future.
The OP was frustrated by nobody buying into their own idea, but they write about it like it was innovative and unappreciated rather than common and naturally limited.
If you've ever had to figure out a weird behavior that was ultimately the result of unsiloed apps just hanging out in each other's data, or had to deal with an uncontrolled Smalltalk object soup, you'll have strong opinions on this idea. They are not positive.
All these environments went away for a reason, and it wasn't their indubitable supremacy. They were great for solo devs, but they break down at a larger scale. (Often, not always, you can find counterexamples, but, again, the market has judged)
And given that we can't even really decide on decent data format for dates (https://en.wikipedia.org/wiki/List_of_date_formats_by_countr...), the view that composability were easy if we just were less siloed is, at best, naive. Even if we managed to somehow achieve it without making a giant code mess.
24 comments
[ 3.1 ms ] story [ 60.1 ms ] thread> The idea of co-designing language and UI
Is because commingling presentation, use cases, and biz logic is highly coupled.
UNIX files have just open, close, read, write, and seek in the simplest form.
The UNIX 'everything is a file' is really just a form of dependency inversion.
I am old enough to remember when the visual languages and UML were supposed to replace programmers with architects with GUIs.
But it ignored that commingling implementation details produced systems that are tightly coupled with implementation details and is difficult to work with.
It seems that these principles are ignored and rediscovered over and over again.
Clean, hexagonal, onion, and ports and adapters architectures being some of the most recent versions.
The UI should depend on the core system and that core should be ignorant of the UI as much as possible.
Adobe has no real reason to expose PS functions that way, and obviously I am talking about ideals. But it is just simple reality that when you tightly couple to implementation details you paint yourself into a corner.
It failed for reasons that are obvious in retrospect. Creating a system where you can embed one app in another requires everyone to agree the interface standard. But the interface is extremely complicated! You have to pass arbitrary data between two applications, and do so quickly. You also have to decide on how much the apps are isolated from each other (getting it wrong creates either massive security failures, or performance that is way too low). You also have to deal with such issues as how to synch the user interfaces!
So any such system is going to be extremely complicated, and getting everyone to agree on standards is almost impossible. What standards do get created are so complicated that they become unusable except for very stereotyped use cases. The other alternative is when a single company forces a standard down everyone's throat. But that creates the problem of vendor lock in, so people tend to reject that!
There are in fact a couple of notable successes. The most obvious is the digital sound market, which btw is a really interesting software market because there is huge competition. There are 4 major DAWs (digital audio workstations), and a number of still used minor DAWs, plus lots of companies that make synths, effects, tools you name it. And they all get sold! A huge amount of this software is in the form of "plugins" apps that run in the DAW as a container. This works because the interaction between plugins is a) similiar to unix pipelines (it forms a directed acyclic graph), so the interation is relatively controlled, and b) there are two standard data interchange methods. One is raw sound data in the form of 32 bit 48 kbs PCM, and the other is midi (a form from the early 80s for controlling synths). The user interface problem was solved in the easiest way... there is no user interface specification, or library, and every app has to roll its own interface (which it does by just using bitmaps... you end up with the most horrribly skeuomorphic designs you've ever seen!). There are actually 3 different standards for interfacing with the DAW, but one VST is dominant. Moreover, because of the simplicity of the data exchange format, the interfaces are actually quite simple to use, if you know about the weird world of audio programming.
The other success story is Microsoft Office. Office is based on MS's tech for creating components, known as com. When you a draw a graph in word or excel, you're actually using a different little app that draws graphs... and you can embed that in your own (windows) app. But COM is extremely complicated, and com components are almost used in apps specifically designed to be part of Office. There's a fairly big ecosystem of these apps (or there was) but its very tightly controlled and integrated by Microsoft.
This was the approach taken by Arexx. I've never used or seen IBM's Rexx outside of the Amiga version, Arexx. While a PC version of Rexx existed, it's obvious why it didn't take off... IBM wanted exorbitant fees per end user, and even higher developer fees. As a result it had no users, and with no users, no software bothered to supported it. Most PC developers likely had no idea it was even a thing.
It was different on the Amiga. Commodore somehow got the rights to including it in their OS, no additional licensing required. As such, there was a ready user base, and thus many apps had Arexx support. You could query data from one app, massage it in the scripting language, then pass it off to another app.
It was a well-loved feature allowing all sorts of automation that wasn't really available anywhere else.
It died.
I don't think there are any real technical of programming language challenges to making this work. It's a problem of economics and sociology.
Economics:
If your doc/app/thing is composed of a multitude of widgets developed by different developers, who pays for those widgets to be developed? If you have to buy them, does that mean you get a doc that you can't send to anyone else unless they have they've paid for the same widgets?
(This problems exists today in the world of music production where there is a thriving plug-in model of third-party VSTs which are wonderful for extensibility but do make it quite hard to share projects with people.)
If that's not the business model... what is? I don't think anyone ever figured out a good way to make money making software like this outside of limited domains.
Sociology:
Conway's law [2] isn't a joke. It's a real observation that for different software components to interoperate, the teams making them need to communicate at least enough to create protocols.
Defining software protocols that are expressive enough to be useful but simple and stable enough to be widely used by completely independent competing organizations is really hard.
Imagine you had to design an API for composable UI widgets. How would you handle text rendering? RTL? Theming? Colors? Layout? It's not intractable, but it's not far from it.
Both:
Then you have to consider that if you have competing organizations trying to make money selling widgets, they don't want their competitors widgets to be successful. They are incentivized to undermine compositionality whenever they can (Microsoft's old "Embrace, extend, and extinguish" [3]).
Also, a fundamental tenet of usability is that the interface as a whole should be consistent and hang together. Users don't want apps that are a patchwork of jarring widgets each with its own style and convention. If you could get a platform like this off the ground, users would still like converge onto a small number of companies that made large sets of widget suites that are designed to compose together harmoniously. The usability benefits of doing so would work like network effects to lead towards platform consolidation.
Software people just dramatically underestimate how hard composability actually is in practice. It doesn't matter if you can shuffle bits between two pieces of code. That's the easy part. The hard parts are all in protocol design, usability, competition, etc.
[1]: https://en.wikipedia.org/wiki/OpenDoc
[2]: https://en.wikipedia.org/wiki/Conway%27s_law
[3]: https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
Nowadays people don't pay but it doesn't mean you can't compose things from open source stuff like material UI.
I think that Bracha's point is that it's always been hard for a user to compose apps and make them consistent etc(although VB was much easier than web stuff). But now the AI will compose them so the promise of VB etc could be fully realised. Power to the people.
Dave Thomas was talking about model generated code and models were a much more precise starting point than the prose we input to an AI.
[1] https://en.m.wikipedia.org/wiki/Cyberdog
[1] https://www.omg.org/technology/documents/vault.htm - under Retired Specifications at the bottom of the page, under Compound Presentation & Interchange
In particular, JavaScript has evolved into a much nicer language than it started out, even more so if you add TypeScript. Yes, there are unfixable gotchas, but they’re well-documented and they’re far less important than some critics think. They rarely get in the way. There are tools that could be better, but they’ve improved a lot, and keep improving.
Also, embedding one app in another visually is a poor fit for mobile devices. There’s only so much screen to work with. Share buttons using common data formats mostly do the job. On the web, the best way of sharing something widely is with a URL. We have iframes but rarely use them.
[1] https://en.wikipedia.org/wiki/Canon_Cat
A lot of legacy stuff there.
It's an obvious idea, sometimes productive, but tends to accumulate incoherence, impedance, and security issues over time so (with exceptions) any given implementation tends to burn itself out eventually.
There are living examples now, there were many examples in the OP's 2005 world, and there will be more in the future.
The OP was frustrated by nobody buying into their own idea, but they write about it like it was innovative and unappreciated rather than common and naturally limited.
All these environments went away for a reason, and it wasn't their indubitable supremacy. They were great for solo devs, but they break down at a larger scale. (Often, not always, you can find counterexamples, but, again, the market has judged)
And given that we can't even really decide on decent data format for dates (https://en.wikipedia.org/wiki/List_of_date_formats_by_countr...), the view that composability were easy if we just were less siloed is, at best, naive. Even if we managed to somehow achieve it without making a giant code mess.
Free as in beer Java or seat by seat workstation license fees — "the market has judged".