This might be useful (maybe) to the author: I am in a group (probably small, but I don’t think negligeble) that gave up on Warp because it doesn’t support standard or custom command completions [0]
Some lesson must surely be drawn from this about incremental adoption.
There's even more under the "Updates archive" expando in that post.
It was a pretty compelling prototype. But after I played with Polyglot Notebooks[1], I pretty much just abandoned that experiment. There's a _lot_ of UI that needs to be written to build a notebook-like experience. But the Polyglot notebooks took care of that by just converting the commandline backend to a jupyter kernel.
I've been writing more and more script-like experiments in those ever since. Just seems so much more natural to have a big-ol doc full of notes, that just so happens to also have play buttons to Do The Thing.
This I am afraid does not feel as the right thing to do. I think the KISS-principle is a good standard to follow and while terminals have to do more than one thing, it should not do everything. Complexity leads to instability. Terminals has to be nimble and not clumsy behemoths like web browsers.
I may disappoint you with the fact that IBM PC-compatible computers have replaced devices of that class. We can only observe certain terminal emulators in some operating systems. There have been many attempts to expand the functionality of these emulators. However, most features beyond the capabilities of VT100 have not caught on (except UTF-8 support). I do not believe that anything will change in the foreseeable future.
I read the whole thing and at first glance, it seems like a whole NIH list of wishes. We already have alternatives to the terminal, but the article have no mentions of them:
- Emacs (inherited from lisp machines?). A VM which is powered by lisp. The latter make it easy to redefine function, and commands are just annotated functions. As for output, we have the buffer, which can be displayed in windows, which are arranged in a tiling manner in a frame. And you can have several frames. As the buffer in a window as the same grid like basis as the terminal emulator, we can use cli as is, including like a terminal emulator (vterm, eat, ansi-term,...). You can eschew the terminal flow and use the REPL flow instead (shell-mode, eshell,...). There's support for graphics, but not a full 2d context.
- Acme: Kinda similar to emacs, but the whole thing is mostly about interactive text. Meaning any text can be a command. We also have the tiling/and stacking windows things that displays those texts.
I would add Smalltalk to that, but it's more of an IDE than a full computing environment. But to extend it to the latter would still be a lower effort than what is described in the article.
I appreciated the Pluto.jl mention! Going from Pluto notebooks that understand data flow to Jupyter notebooks where you have to tell the computer which order to run the cells in is always baffling to me. Why doesn't Jupyter know the run order and dependencies already? The way Pluto handles dependencies between code cells is really just so nice.
Re "stage 2: persistent sessions", iTerm2 already does this. The PTYs are owned by background processes independent of the GUI application. So iTerm2 can upgrade the application without interrupting the running terminal sessions.
I dread reading these articles written by people who perceive the command-line interface / terminal / shell to be this archaic system, and propose "modernizing" it by cramming whatever UI/UX they're most familiar with.
The last thing a command-line terminal needs is a Jupyter Notebook-like UI. It doesn't need to render HTML; it doesn't need rerun and undo/redo; and it definitely doesn't need structured RPC. Many of the mentioned features are already supported by various tooling, yet the author dismisses them because... bugs?
Yes, terminal emulators and shells have a lot of historical baggage that we may consider weird or clunky by today's standards. But many design decisions made 40 years ago are directly related to why some software has stood the test of time, and why we still use it today.
"Modernizing" this usually comes with very high maintenance or compatibility costs. So, let's say you want structured data exchange between programs ala PowerShell, Nushell, etc. Great, now you just need to build and maintain shims for every tool in existence, force your users to use your own custom tools that support these features, and ensure that everything interoperates smoothly. So now instead of creating an open standard that everyone can build within and around of, you've built a closed ecosystem that has to be maintained centrally. And yet the "archaic" unstructured data approach is what allows me to write a script with tools written decades ago interoperating seamlessly with tools written today, without either tool needing to directly support the other, or the shell and terminal needing to be aware of this. It all just works.
I'm not saying that this ecosystem couldn't be improved. But it needs broad community discussion, planning, and support, and not a brain dump from someone who feels inspired by Jupyter Notebooks.
Great thought provoking article! Indeed, typing commands on the command line feels primitive like typing code into interactive interpreters (python, irb, etc.). Those are primitive REPLs.
With lisp REPLs one types in the IDE/editor having full highlighting, completions and code intelligence. Then code is sent to REPL process for evaluation. For example Clojure has great REPL tooling.
A variation of REPL is the REBL (Read-Eval-Browse Loop) concept, where instead of the output being simply printed as text, it is treated as values that can be visualized and browsed using graphical viewers.
Existing editors can already cover the runbooks use case pretty well. Those can be just markdown files with key bindings to send code blocks to shell process for evaluation. It works great with instructions in markdown READMEs.
The main missing feature editor-centric command like workflow I can imagine is the history search. It could be interesting to see if it would be enough to add shell history as a completion source. Or perhaps have shell LSP server to provide history and other completions that could work across editors?
- https://arcan-fe.com/ which introduces a new protocol for TUI applications, which leads to better interactions across the different layers (hard to describe! but the website has nice videos and explanations of what is made possible)
I tried to find a terminal which can shows images, videos, and other media. I know it's not a UNIX way, but sometimes it can help. There are multiple options, but they didn't click in me.
But just showing a browser like Jupyter would be very useful. It can handle a wide variety of media, can easily show JS heavy webpages unlike curl, and with text option to show text based result like w3m but can handle JS, it will be more useful.
browser google.com/maps # show google map and use interactively
browser google.com/search?q=cat&udm=2 # show google image result
browser --text jsheavy.com | grep -C 10 keyword # show content around keyword but can handle JS
vim =(browser --text news.ycombinator.com/item?id=45890186) # show Hacker News article and can edit text result directly)
My personal opinion-psuedoterminals should be enhanced to provide some mechanism for sending out-of-band data, like the existing TIOCPKT but on steroids. Maybe something like… if both ends support it, they can exchange JSON-RPC messages over this out-of-band channel, with it being possible to discover if the other side supports this or not. Possibly this is just a new TIOC* ioctl
Why? Well one reason is escape sequences are really limited and messy. This would enable everyone to gradually and backward-compatibly transition to a more modern alternative. Once you have a JSON-RPC channel, the two ends can use it to negotiate what specific features they support. It would be leveraging patterns already popular with LSP, MCP, etc. And it would be mostly in userspace, only a small kernel enhancement would be required (the kernel doesn’t have to actually understand these JSON-RPC messages just offer a side channel to convey them).
I suppose you could do it without any kernel change if you just put a Unix domain socket in an environment variable: but that would be more fragile, some process will end up with your pty but missing the environment variable or vice versa
Actually I’d add this out-of-band JSON-RPC feature to pipes too, so if I run “foo | bar”, foo and bar can potentially engage in content/feature negotiation with each other
I had thought of a idea in a operating system design, which has many similarities to this but also many significant differences. The rest of the operating system would also be designed to support this, and the REPL would also be integrated with editors (like they show) and with a zoned spreadsheet. There would also be transactions (not the same as described there, but there is a similarity), and also capabilities (usable for security and many other things), and possibility to move objects between windows (this also allows it to work better with GUI; e.g. you can write a loop and use some combination of keys with mouse to drag a command button from a GUI window into your text in order to execute that command button within the loop; or you can copy data displayed in a table in another program in order to perform calculations on it or write a conversion to other format to then copy into another program, etc). However, tabs and panes would be handled by the window manager. However, it would use a binary format (a kind of TLV format), and it would not use CSS and HTML and Unicode. For rich text and other things there would also be a Reveal Codes function in order to reveal (and modify) the codes.
I also have lots of ideas, not all that different from the goals of some of these... but there's something critical to remember. The terminal survives as it is, because so many tools were built to run on them. They are only becoming more cross platform and feature rich, albeit slowly and conservatively.
Maintaining a high level of backwards compatibility while improving the user experience is critical. Or at least to me. For example, my #1 fristration with neovim, is the change to ! not just swapping the alt screen back to the default and letting me see and run what I was doing outside of it.
We generally like the terminal because, unlike GUIs it's super easy to turn a workflow into a script, a manual process into an automated process. Everything is reproducible, and everything is ripgrep-able. It's all right there at your fingertips.
I fell in love with computers twice, once when I got my first one, and again when I learned to use the terminal.
> It does this using (mostly) standard features built-in to the terminal and shell (a custom DCS): you can read their explanation here. It's possible to do this less invasively using OSC 133 escape codes; I'm not sure why Warp didn't do this, but that's ok.
Independent of the rest, I would love for more terminal emulators to support OSC 133.
I wish people would stop back-porting new features into vt terms, since those “features” gradually leak into terminal programs, which start spewing garbage in older terminals at the next update. Instead of having a standard, we get ESC[whatever-the-fuck-i-feel-like;, but with vt220 characteristics.
Missing out on inline images and megabytes of true-color CSI codes is a feature, not a bug, when bandwidth is limited.
If you want jupyter, we have jupyter. If you want HTML, we have several browsers. If you want something else, make it, but please don’t use vt220 codes and call it a terminal.
The article is just wish-listing more NIH barbarism to break things with. RedHat would hire this guy in a heartbeat.
Effectively you have to use tmux or screen as a condom at that point - programs that refuse to use terminfo/termcap will spew whatever xterm-like escapes they hardcode at tmux/screen, which then uses terminfo/termcap to translate to your actual terminal.
My biggest gripe with it is that it quickly ends up becoming an actual production workload, and it is not simple to “deploy” and “run” it in an ops way.
Lots of local/project specific stuff like hardcoded machine paths from developers or implicit environments.
Yes, I know it can be done right, but it makes it sooooooooo easy to do it wrong.
I think I can’t not see it as some scratchpad for ad-hoc stuff.
51 comments
[ 3.3 ms ] story [ 62.3 ms ] threadSome lesson must surely be drawn from this about incremental adoption.
[0] https://github.com/warpdotdev/Warp/issues/1811
There's even more under the "Updates archive" expando in that post.
It was a pretty compelling prototype. But after I played with Polyglot Notebooks[1], I pretty much just abandoned that experiment. There's a _lot_ of UI that needs to be written to build a notebook-like experience. But the Polyglot notebooks took care of that by just converting the commandline backend to a jupyter kernel.
I've been writing more and more script-like experiments in those ever since. Just seems so much more natural to have a big-ol doc full of notes, that just so happens to also have play buttons to Do The Thing.
[1]: https://marketplace.visualstudio.com/items?itemName=ms-dotne...
https://commons.wikimedia.org/wiki/File:DEC_VT100_terminal.j...
I may disappoint you with the fact that IBM PC-compatible computers have replaced devices of that class. We can only observe certain terminal emulators in some operating systems. There have been many attempts to expand the functionality of these emulators. However, most features beyond the capabilities of VT100 have not caught on (except UTF-8 support). I do not believe that anything will change in the foreseeable future.
- Emacs (inherited from lisp machines?). A VM which is powered by lisp. The latter make it easy to redefine function, and commands are just annotated functions. As for output, we have the buffer, which can be displayed in windows, which are arranged in a tiling manner in a frame. And you can have several frames. As the buffer in a window as the same grid like basis as the terminal emulator, we can use cli as is, including like a terminal emulator (vterm, eat, ansi-term,...). You can eschew the terminal flow and use the REPL flow instead (shell-mode, eshell,...). There's support for graphics, but not a full 2d context.
- Acme: Kinda similar to emacs, but the whole thing is mostly about interactive text. Meaning any text can be a command. We also have the tiling/and stacking windows things that displays those texts.
I would add Smalltalk to that, but it's more of an IDE than a full computing environment. But to extend it to the latter would still be a lower effort than what is described in the article.
The last thing a command-line terminal needs is a Jupyter Notebook-like UI. It doesn't need to render HTML; it doesn't need rerun and undo/redo; and it definitely doesn't need structured RPC. Many of the mentioned features are already supported by various tooling, yet the author dismisses them because... bugs?
Yes, terminal emulators and shells have a lot of historical baggage that we may consider weird or clunky by today's standards. But many design decisions made 40 years ago are directly related to why some software has stood the test of time, and why we still use it today.
"Modernizing" this usually comes with very high maintenance or compatibility costs. So, let's say you want structured data exchange between programs ala PowerShell, Nushell, etc. Great, now you just need to build and maintain shims for every tool in existence, force your users to use your own custom tools that support these features, and ensure that everything interoperates smoothly. So now instead of creating an open standard that everyone can build within and around of, you've built a closed ecosystem that has to be maintained centrally. And yet the "archaic" unstructured data approach is what allows me to write a script with tools written decades ago interoperating seamlessly with tools written today, without either tool needing to directly support the other, or the shell and terminal needing to be aware of this. It all just works.
I'm not saying that this ecosystem couldn't be improved. But it needs broad community discussion, planning, and support, and not a brain dump from someone who feels inspired by Jupyter Notebooks.
With lisp REPLs one types in the IDE/editor having full highlighting, completions and code intelligence. Then code is sent to REPL process for evaluation. For example Clojure has great REPL tooling.
A variation of REPL is the REBL (Read-Eval-Browse Loop) concept, where instead of the output being simply printed as text, it is treated as values that can be visualized and browsed using graphical viewers.
Existing editors can already cover the runbooks use case pretty well. Those can be just markdown files with key bindings to send code blocks to shell process for evaluation. It works great with instructions in markdown READMEs.
The main missing feature editor-centric command like workflow I can imagine is the history search. It could be interesting to see if it would be enough to add shell history as a completion source. Or perhaps have shell LSP server to provide history and other completions that could work across editors?
- https://arcan-fe.com/ which introduces a new protocol for TUI applications, which leads to better interactions across the different layers (hard to describe! but the website has nice videos and explanations of what is made possible)
- Shelter, a shell with reproducible operations and git-like branches of the filesystem https://patrick.sirref.org/shelter/index.xml
But just showing a browser like Jupyter would be very useful. It can handle a wide variety of media, can easily show JS heavy webpages unlike curl, and with text option to show text based result like w3m but can handle JS, it will be more useful.
Why? Well one reason is escape sequences are really limited and messy. This would enable everyone to gradually and backward-compatibly transition to a more modern alternative. Once you have a JSON-RPC channel, the two ends can use it to negotiate what specific features they support. It would be leveraging patterns already popular with LSP, MCP, etc. And it would be mostly in userspace, only a small kernel enhancement would be required (the kernel doesn’t have to actually understand these JSON-RPC messages just offer a side channel to convey them).
I suppose you could do it without any kernel change if you just put a Unix domain socket in an environment variable: but that would be more fragile, some process will end up with your pty but missing the environment variable or vice versa
Actually I’d add this out-of-band JSON-RPC feature to pipes too, so if I run “foo | bar”, foo and bar can potentially engage in content/feature negotiation with each other
Maintaining a high level of backwards compatibility while improving the user experience is critical. Or at least to me. For example, my #1 fristration with neovim, is the change to ! not just swapping the alt screen back to the default and letting me see and run what I was doing outside of it.
We generally like the terminal because, unlike GUIs it's super easy to turn a workflow into a script, a manual process into an automated process. Everything is reproducible, and everything is ripgrep-able. It's all right there at your fingertips.
I fell in love with computers twice, once when I got my first one, and again when I learned to use the terminal.
Independent of the rest, I would love for more terminal emulators to support OSC 133.
Missing out on inline images and megabytes of true-color CSI codes is a feature, not a bug, when bandwidth is limited.
If you want jupyter, we have jupyter. If you want HTML, we have several browsers. If you want something else, make it, but please don’t use vt220 codes and call it a terminal.
The article is just wish-listing more NIH barbarism to break things with. RedHat would hire this guy in a heartbeat.
My biggest gripe with it is that it quickly ends up becoming an actual production workload, and it is not simple to “deploy” and “run” it in an ops way.
Lots of local/project specific stuff like hardcoded machine paths from developers or implicit environments.
Yes, I know it can be done right, but it makes it sooooooooo easy to do it wrong.
I think I can’t not see it as some scratchpad for ad-hoc stuff.