81 comments

[ 2.4 ms ] story [ 339 ms ] thread
Fast is good but given that any terminal is fast enough to type commands and scroll around, convenient is even better. Example: I went with tilix months ago and eventually switched back to Gnome terminal because tilix froze sometimes. Furthermore tilix doesn't have a way to bind commands to a profile to run them when opening a terminal with that profile. The freeze is of course 99% of my switching back.
Never experienced any freezes with tilix and I use it daily for years now. It's abandonware though.
It's still GTK 3 using VTE and so doing the drawing with Cairo. It (VTE) has a lot of optimizations that made sense when we didn't have working GPU drivers.

But for GTK 4, we can do things much differently and faster.

alacritty is fast but kitty is faster. glad to see us racing to the top in this space.
They're both leaving practical performance on the table by not implementing VT420 (or better) control sequences, though. So whatever gains you get from them are lost by applications needing to do more work. For example, tmux uses DECLRMM to restrict scrolling to the active part of horizontally split panes, and as neither Alacritty nor Kitty implements DECLRMM tmux will have to do more work to get the same effect.

The author of Zutty has a writeup about it: https://tomscii.sig7.se/2020/12/A-totally-biased-comparison-...

(comment deleted)
as a contented xterm user, none of these "modern gpu accelerated terminals" really excited me. But the care shown in correct output by the author really grabbed my interest. I probably won't change, as I said I like xterm, but I am going to try it out.
Looking at this, I’m reminded of early clang, which was often 5 or more times faster at compiling C++ than gcc, while handling maybe 95% of C++ code.

While clang is still in my experience faster than gcc, it turns out handling that last 5% correctly can knock a lot off your performance. Of course with terminals it might turn out you don’t need that last 5%, or you really can get all the way and keep the speed, but this isn’t the first project dedicated to making “the fastest terminal”.

The way I see it, modern terminal emulators are kind of like multimedia applications, they've got a frame budget. Today a lot of them don't necessarily spend that budget wisely. I think there is a legitimate point here.

Compilers could be faster, but at this point they've become optimal enough that a lot of the improvements would be trade-offs for something else. In my opinion, not so for terminals: I'm pretty sure you could analyze the amount of time spent rendering in a terminal emulator and come to the conclusion that a lot of it is essentially wasted, with exception of a few more modern terminal emulators that considered this more deeply. This is in part due to the nature of the outputs of terminal emulators vs compilers; like for high throughput, it would be good if the terminal emulator spent as little time as possible processing things that won't ever actually make it to a frame. Compilers are mostly not interactive though, and don't have the luxury of deciding to simply not do something because nobody is looking.

> but at this point they've become optimal enough that a lot of the improvements would be trade-offs for something else.

I don’t think that there’s any evidence that this is true. In fact, the performance of the Rust compiler has continually improved over the last five years while _adding_ new features, and not by removing anything. <https://perf.rust-lang.org/dashboard.html>

All a C++ compiler needs to do to get similar results is to have a team that does nothing but performance work, and is actually rewarded for it.

Yeah, I was talking about C++ specifically. I am extremely skeptical you can get that much better for C++ without sacrificing performance, correctness, etc.

Rust is different. It has a completely different design for translation units and modules that C++ can't adopt without hard breaking changes, due to e.g. the way its macro system works.

There are teams at all of Google, Microsoft, et al that spend time on C++ build performance. At this point, it's not even necessarily just about faster build times. Their builds could be measured in environmental impact at that scale.

I feel like there's room for C++ without headers or includes — a completely clean break.
That would be pretty interesting. Not quite as interesting as programming in Rust though :)
That’s what C++20 modules are designed for?
Mold is substantially faster than other linkers [0], so that's evidence there's still room for improvement in the C++ compiler space. It does help that there are a lot of linker features that are only used in special environments (like bare-metal systems).

[0]: https://www.gem5.org/project/2023/02/16/benchmarking-linkers...

Mold really is a big improvement, I'll give you that. (I'm a big fan of mold, to the point where I happily joined the Github Sponsors when it went more open recently.) But, the compilation stage is a tough nut to crack.
C++ also has modules.
For right now, this doesn't seriously help the problem, for multiple reasons. C++ modules actually tend to reduce compilation performance, since modules need to be processed before their dependents, whereas normal translation units are fully independent of each-other.
It surely does, as per Microsoft remarks doing import std; is faster than #include <iostream>.

Since VC++ modules support got good enough, that all my side projects only use modules.

Importing standard libraries a bit faster is not really the greatest concern of large C++ projects.
Office team is already on that path, I suppose they fit the bill.
Is rustc is backward-compatible, does it typically compile .rs files written 1 year ago? 5 years ago?
Rust has really great support for backward–compatibility.

A few breaking changes to the language have been made over the years (such as the introduction of async/await), resulting in several different editions of the language. The compiler supports all editions equally, and different crates can even be in different editions, even if they are linked into the same program. That means that code written for Rust 1.0 still works perfectly, and there is no need to update it to the latest edition in order to continue using it even in programs that are written in the latest edition.

(comment deleted)
Is it handling the 5% correctly or is it just a lot more optimization passes that make the difference?
That's a good question. I'd personally go with the last 1/2 of your sentence.
I've spent a lot of time profiling clang when compiling the Linux kernel.

For large C codebases, compile time is dominated by lexing and parsing, not optimization passes.

Headers have become too crufty; they tend to only grow over time, resulting in an amplification attack on the lexer and parser.

Lazy parsing is maybe interesting. There's a hazard related to the mangling of lambdas. Carbon has an interesting idea of not constructing an AST.

I think we also need tools to help us automate header refactoring.

That's a surprise to me. That must be one epic mass of header files.

It reminds me of the massive Fast kernel headers patchset by Ingo Molnar. I hope it got in. I don't know anything about it, just marvelled at the scope of it.

The profiler frequently surprises.

Imagine 30 years of headers. They tend to grow and grow and become a tangled mess.

Ingo's patch set did not.

I asked what was being done to automate this so that we wouldn't backslide.

I received no response.

That said, I think that work is fascinating. I have an intern starting Monday that's going to start research into automating this.

Author of the code in question here ...

Not sure how this got off terminals and on to compilers, but here we are.

Having written the profiler I used to optimize this (Sysprof), and a large portion of the OpenGL renderer that GTK uses, I know a thing or two about what code runs in the hot path from VTE. So I could target something that does enough of that to see what should be possible as a best case / upper bound.

That's why I'm fairly confident that adding the rest of the features that VTE has (which is one of the most comprehensive terminal emulator code bases out there) wouldn't affect this particular test case.

That said, I have no interest in doing that. I'd rather just go make VTE fast at drawing and call it a day. I have literally dozens of other projects in GNOME that require my time.

Hi!

I hope you are right, and also someone else will pick this up -- I've just seen an awful lot of projects over the years that provide a huge speedup for 90-95% of the functionality of other products, and very few manage to keep the speedup once they hit full functionality.

I really doesn't matter to be honest, because this was just to inform the theoretical upper bounds as part of my work to make VTE faster.
Although people wonder why anyone cares about performance in terminal emulators, it's more about caring about inefficiency in terminal emulators. There are some legit cases where performance actually does come into play, like with high-throughput. Sometimes it's not even necessary, such as when terminal emulators do more work than they should processing data that won't be displayed; those things have the potential to impact the experience even when there is plenty of CPU and GPU to spare, like on a desktop machine. However, it's not just performance. It's also energy usage, particularly on small things like the Pinephone.

Linux being an OS where the terminal is as powerful as it is, it's really good to see a high level of attention to detail going to terminal emulators finally. Terminal emulators should be extremely robust and extremely efficient; and finally, they're getting pretty close.

I never really cared, but when I first dropped into a proper terminal in Ubuntu many years ago using ctrl+alt+n, or whatever the combo is, it really felt like the characters were appearing on the screen before I even touched the key. That’s when I learned emulators have a noticeable lag. I still don’t care much, but it sucks to know it’s there, no matter how much I spend on hardware.
Ctrl-Alt-F3 to console and Ctrl-Alt-F2 to go back to the GUI?
Where the GUI is has changed over the years/by distro, so I use Ctrl+Alt+(right arrow) to cycle between them until I find it.
xterm has no built in lag (your compositor will introduce some, but it's possible to disable compositing if you use Xorg). Kitty can be configured to have practically identical lag to xterm.
I've never used anything other than the default terminal in whatever system I'm working on, and can't say that I've ever noticed or thought that it wasn't "fast enough."

I started using computers with a vt100 terminal back in the 300/1200/2400 baud days though, any modern terminal feels basically infinitely fast in comparison.

One thing that you could definitely miss is the gradual creep of latency. While throughput on terminal emulators is certainly infinitely faster than 300 baud physical terminals, latency has suffered a bit over time: buffering (sometimes to improve throughput) has added a frame here and some milliseconds there. Granted, I have Never used a physical terminal, but I believe some of them had enough circuity to locally update the cursor during line editing and whatnot, and if that was the case, in theory they should have excellent latency characteristics.

On one hand, these latency figures are so small that they're usually difficult to directly perceive; who can even notice the difference between e.g. 25ms and 30ms of latency? That aside though, I believe it likely still negatively impacts the experience. Personally, it feels extremely pleasing to use a pre-compositing desktop environment on a CRT display; the lack of latency is visceral. I am so used to modern machines that it typically feels like it's updating before I have even performed the action.

I think that modern terminal emulators and text editors in modern desktop environments on modern computers with modern peripherals run the risk of having latency bad enough that it actually, if only subtly, impacts typing. USB polling rates, relatively high latency 60hz panels, buffering from compositing, it all does add up over time for sure. Add SSH to the mix and it could legitimately make the difference between something feeling quite usable and something that is irritating.

Latency lost can never be recovered or never be parallelized.
It seems like he's not even sharing the code, which is unfortunate if Alacritty or others want to learn from this. I guess that's a personal choice, but is it in the spirit of fosstodon (the mastodon instance he's on.)?
I already maintain two dozen GNOME projects. I wrote a huge amount of GTK's OpenGL renderer, the profiler we use on GNOME, GNOME Builder, and a huge amount of other parts of GNOME. I've even started to fix the library that people will actually use (VTE) instead of fragmenting communities with some other project.

My commitment to "FOSS" is not up for debate.

I've written in multiple places what it takes to get stuff that fast. If that isn't enough for any of those projects, I'm happy to take money in exchange for my time.

This reminds me of the kerfuffle Casey Muratori caused last year when he dared suggest the windows terminal could be faster.

https://twitter.com/cmuratori/status/1523024109831041024

https://github.com/cmuratori/refterm

I feel like you are really under-selling that entire drama. It started with Casey pointing out some slow terminal behavior[0], to which he received unreasonable flack, including this choice quote,

>I believe what you’re doing is describing something that might be considered an entire doctoral research project in performant terminal emulation as “extremely simple” somewhat combatively…

Casey then spent a weekend and produced a couple thousand line terminal emulator which seemed feature complete(? probably missing some bells and whistles), but blew away Windows performance.

A year later, Microsoft released a patch, roughly incorporating some of Casey's suggestions, and did not credit him for anything[1].

Edit: This[2] seems to be the much bigger Hacker News discussion about the topic instead of [1]. If nothing else, [2] has a repeatedly edited post by the Microsoft author where attitudes are on full display.

[0] https://github.com/microsoft/terminal/issues/10362

[1] https://news.ycombinator.com/item?id=31372606

[2] https://news.ycombinator.com/item?id=31284419

To give a more complete story, they did belatedly credit him and others, in a later edit. Still a very poor showing from the relevant people at Microsoft.
Wait, what? I read through that entire discussion. I do agree that the quote you pulled out was not ideal, but that's the only thing in there that's even remotely abrasive. Skyline in particular took great pains to provide context of why the performance issue was the way it was.

Where's the flak, exactly? (FYI, no "c" in "flak")

Except without any kerfuffle.
I may write my own but wayland, pure CPU rendering, assembly (x86_64/risc-v)... but... I am still on other projects.
Meanwhile, all GNOME-based terminals are still capped at 40fps:

https://github.com/GNOME/vte/blob/master/src/vte.cc#L10770

Well, it fits in with the bunch of folks that deem system tray icons, application theming, fractional scaling or window title bars unnecessary to arbitrarily limit their terminal's performance, while the competition has no such penalty.
It sounds like you're pretty biased, and that's fine. You do you.

But I'm pretty sure that code pre-dates reliable access to vertical sync from applications. Either via XProperty notifications from the compositor, XPRESENT, or actual glSwapBuffers() and/or equiavalents.

First we make it fast. Then we remove the FPS restriction. I would be very surprised if I don't manage to nuke that as part of this work.
I once played a esoteric video game where you had to interact with a terminal inside a 3d world as part of the gameplay mechanic. It was implemented in unity, and it featured motion blur when a lot of output was generated. Since then I always wonder if text editors and terminals could actually greatly benefit from motion blur whenever text is moving.

The purpose of motion blur is first to make things seems smoother, but mostly to indicate how things are moving on the screen with lower framerates. I think with motion blur you could make text jump way further frame by frame while understanding how text was moving. If you have a word that is bold on screen, you could make it jump half a screen with within a single frame while immediately understanding where it went. Wouldn't that help navigating e.g. huge logs (esp. if they're colored) faster while still having a sense of how far you moved frame by frame?

If your terminal outputs so much that nothing of the screen is the same frame by frame, you could also still get a feeling of how much is actually outputted, by how blurry the screen is.

Never got around implementing it though, curious what you think!

Could you be talking about Event[0] by any chance?
Motion blur can help compensate for low framerates if it's (approximately) physically accurate, i.e. temporally oversampled. This is how 24fps movies can look tolerable despite the ultra-low framerate: each frame provides more information about how the contents are moving than just a single unfiltered sample, because you can follow the motion blur trails.

This is complicated by computer graphics being gamma corrected. If you average multiple temporal samples together in sRGB colorspace, brightness will vary unrealistically with motion speed. You need to calculate the motion blur in linear colorspace then convert to gamma colorspace before displaying it. And for best results, the image to be blurred should be HDR, with the blurred version mapped to the lower dynamic range of the screen. This best replicates in-camera motion blur, giving you clear trails behind bright points.

However, any deliberately introduced motion blur is inferior to just increasing framerate. Your eyes already provide motion blur. Anything added by simulation or camera is unrealistic additional motion blur on top of this.

I can't really speak to the idea of added motion blur; aside from bias - I deplore it.

However, with high refresh rate displays ~120Hz+ (and today's performant terminals), I can read logs incredibly well. I can pick phrases out of absolutely constant, non-pausing, streams.

These displays are typically touted as for twitch-like FPS gamers, but they're honestly very useful for consuming a large amount of logs.

There's of course natural blur that comes with this - the quality of the panel decides how clear.

Using Wayland probably helps, output is synchronized by default IIRC.

Glimpses of very fast output can still be read by persistence of vision while staring at one part of the screen (you can still get a sense of what's being compiled at the moment, for example), which added motion blur would defeat. Maybe a terminal feature like `less -w' where a status column shows a (potentially motion-blurred) indicator of the amount scrolled in each frame could make the best out of both use cases.

Related personal observation: A higher refresh rate increases the max scroll speed where text tracked with the eyes is clear enough to read, but decreases the legibility of fast output read by persistence of vision. When I tested this – the store employee was rather puzzled at this use case – 60Hz seemed to be optimal. At 120Hz or higher, scrolling text was (edit) less comprehensible at the same output rate. This applied to various panels and drive modes.

I've never used a high refresh-rate CRT, but I surmise they don't t differ here, since CRT flicker only reduces motion blur that is itself caused by eyes smoothly pursuing onscreen objects made of stationary frames.

Edit: Speaking of scrolling UX, I wish browsers had the option to "overscroll" when paging down, so the unread portion always starts at the top of the screen, as implemented in https://news.ycombinator.com/item?id=36825027. You can make less do this with `-c' (not at all evident from the documentation). A userscript, perhaps...

I turn off motion blur in every game, would hate it on the terminal lol
Happy to see more folks working on fast terminal emulators!

Earlier this year, I went through every single terminal emulator in Arch's repos and tested using them while compiling AOSP (Android). AOSP's Soong build system does not produce a lot of scrollback, but it does overwrite a line containing the progress ~150k times (once per source file). Alacritty was the only one that was both responsive and didn't eat up a CPU core (staying under 10%). Wezterm, Kitty, Foot, and Konsole stayed responsive, but used 100% CPU on a single core, causing a measurable impact to the build speed (though Konsole 23.08 seems to have regressed and now sometimes locks up). Most of the others became unresponsive or had multiple second delays trying to do anything (eg. ^C, opening new tab, etc.).

What I was trying to show, and also why I have no interest in creating something out of Termkit, is just that we have a lot of low hanging fruit that can still be fixed. I'd rather do that work as part of VTE so all the applications I use benefit (including GNOME Builder, which I wrote and maintain).

This was a fairly straightforward implementation of VT parsing and uses the retained render tree in GTK properly. There is no need to sacrifice CPU for GPU or vice-versa. You can literally do better in both directions while rendering at full frame rates.

Why care about terminal emulators speed? I never cared about anything (let alone terminals) to go above 30 fps. I've heard extra FPS can help in professional gaming but I can't even imagine a case when speed of any existing terminal emulator would be not enough (for what?). Surely its nice to make any app as efficient and responsive as physically possible (as long as this doesn't harm anything) but I still feel curious about what, besides pure perfectionism and competition, actually motivates people in this case.
I recently upgraded from 60 Hz to 160 Hz monitors and just regular mouse movement/scrolling/etc. is so much smoother. Is it necessary? No, but I won't willingly go back to 60 Hz, let alone 30 Hz.
30 hz 4k with shitty USB-C dongles is a terrible experience
Exactly my work setup at the moment. Amazing how quickly my excitement at the new 4k monitor was quelled.
I've done the same and was pretty dead set on it a while for gaming. And then I got a steam deck which can sometimes dip into the 30s or even 20s and I realized that it does feel jankier but it's definitely an acceptable trade off for something like the mobility of a deck. I'll even crank up graphics on my gaming desktop sometimes now just for eye candy even though it runs closer to 30-45fps.
High stable fps is important for multiplayer shooters, and presumably other multiplayer or difficult genres where smoothly tracking objects with your mouse is part of the core gameplay.

Every time I upgrade my desktop I notice measurably higher stats.

> I never cared about anything (let alone terminals) to go above 30 fps.

I find that a bit hard to believe. Try setting your monitor's refresh rate to 30hz instead of 60hz, and the difference is huge. Going from 60 to any higher is also a very obvious improvement to the user experience (not power usage).

But I agree about the terminal emulator case. I've personally never had an issue with terminal performance on Linux, but someone in the comments mentioned that building AOSP in a slow terminal actually slows the build down due to it taking up an entire core just for rendering the display. Although, I think that's an extreme edge case.

Well, yes, the actual monitor is the only exception. Although low refresh rate never looked different than high to me, it somehow felt worse. On CRT monitors I have tried 60 Hz (looks Okay but feels unpleasant), 85 Hz (totally Ok), 100 Hz (no difference from 75 Hz). On LCD displays I tried 60 Hz (perfectly Okay, what I consider the norm), 75 Hz and 100 Hz and both felt no different from 60 Hz - I would never be able to distinguish. I may confuse 75 with 85 and 100 with 120 - I can't remember what exactly that was clearly. So a 75-85 Hz CRT or a 60 Hz LCD feels like what I need. Less is worse, more makes no difference.

But this is about the physical refresh rate of the display. What I meant in the previous comment is about the rendering FPS a program would produce.

For me, keypress latency. I've used xterm for so long that whenever I end up in gnome-terminal, there's just enough delay that it throws me off while typing.

And yeah I find it kind of funny this post is by a gnome developer when gnome-terminal is my main example for this.

How did you learn to press keys faster than 30 times per second?
You're thinking bandwidth, not latency. The lag is detectable with a single keystroke.
When you are used to very low latency, everything else feels sluggish and more indirect. The immediacy of 1980s hardware induces a sense of realness that just isn’t quite there with higher latency. It’s like direct manipulation vs. going through a medium.
Back in the day, terminal emulators were fast enough that they were up to date by the time each 50Hz/60Hz period came around - why on earth are they now so much slower?

A couple of days ago I spent a while trying to figure out why a piece of hardware appeared to be outputting serial data in bursts. After eliminating issues with the underlying hardware and buffering in the USB stack, it (too slowly) dawned on me that the issue was entirely due to how slow terminal emulators have become (this was on a fastish PC).

Updating a screenful of text at 60/120Hz, why/how is it so much slower these days? (and what benefits have we gained in return for the slowdown?).

Honestly, all I ask from a terminal emulator is to have a slick and fast search UX (similar to item on macOS)
I've tried every "accelerated" terminal app that I see, yet I find I never perceive any improvement in response, only the negative effect of losing the helper UI of menus and such compared to my (now ~10 year usage) usual term of Sakura.

Kitty came closest to being usable for me, but still it involved throwing away ease of access to various things to have a boost to performance that was only something I was told I was getting, not something I was experiencing.

> "Just going to put it out there because I don't intend to do anything with it, but I have created a terminal emulator that is twice as fast as the closest GPU-based renderer I've found (at least on Linux) which was Alacritty."

> "I don't care too much because creating your own terminal is like 20 lines of code these days. People who really care can just create one as easy as configuring an existing one."

These sentences sounds conceited and arrogant.

"If you cant do it you are trash" vibes
Just look at Windows Terminal - which is pretty damn fast.