On a slightly different level, there's been several periods (mostly just after the introduction of new CPUs, Pentium Pro era especially) where multiple compilers produced faster code with "size" optimizations (-Os) rather than the speed optimizations (-O3), which was so counter-intuitive some folks literally wouldn't discuss the possibility.
I had one where tuning the code to fit the cache size (by using a simpler, smaller, "less efficient" algorithm) led to a ludicrous performance gain ... and several burnt 85Mhz Sparc CPUS, they expected pauses occasionally on the data bus apparently.
It is not at all counterintuitive. The compiler has no model of CPU cache so it is completely oblivious of the effects of various optimizations on caching performance.
Most of the time the optimizations work as advertised but there is no problem contriving examples that show ridiculous improvements by removing speed optimization.
There are various limits that have huge effect on your code performance. For example, if your inner application loop does not fit instruction cache its performance might be impaired. If you can reduce the size of the code, ironically by removing speed optimizations, this might just be what is needed to dramatically speed it up.
On macro scale years ago I have been using Gentoo and, generally, compiling almost everything with size optimization had very positive performance improvement on a platform that has been deficient in RAM.
I guess "size" improvement should not be thought as opposite of "speed" improvement but rather an alternate way to possibly improve performance. The compiler does not know which alternative may or may not bring effects.
Y'kno, I agree with the point, with Steven, with Casey, and with other people who are saying "We can have awesome performance without having horribly complicated code", but I find that people who espouse these kind of views are always so curmudgeonly? They just constantly have this view of "I"m right, everyone else is wrong, everyone else is making dumb excuses". If you just let people have their egos you'd probably be a lot more successful in making software more performant.
Since I'm in a similar boat to them in regards to my feelings about the state of modern software, I think they're like this because it has been going on for over a decade now and it just keeps getting worse.
Note that this whole twitter thread is in reference to Casey lodging an issue on GitHub about how slow Windows Terminal is, wherein he was told that his simple suggestion for making it fast would be a PhD level thesis worth of work. He was being polite and explaining things to them until then, but after that statement he was so disgusted he closed the issue and created refterm to prove his concept[0]. Then a bunch of people came out of the woodwork to make excuses for why it still couldn't be done. It's a fucking terminal emulator, a piece of 80s technology[1] that mimics a 60s technology, running in 2021, and people still think it makes any sense at all that it could be slow!?
[0] which is not anything groundbreaking, it's pretty much the obvious way to do things. He has intentionally done no optimization since it's designed to serve as a minimum performance benchmark.
[1] I'm guessing here, since I can't readily find an answer to when the first terminal emulator was released.
FWIW, filling a screen with monospaced pixel-font text (plus colors and stuff) takes single-digit microseconds on a modern GPU once everything is in on-device memory. Fancier fonts or distance-field-based antialiasing is a few times that.
All the rest of the time is bookkeeping and memory transfers.
This is exactly Casey's point, and something I've been ranting about a lot myself: Programmers these days have mental models of performance that are often "off" by three orders of magnitude or more! To clarify: Not a factor of three, but incorrect by thousands.
My older RTX 2080 Ti has main memory bandwidth (not on-chip cache bandwidth!) of 616 gigabytes per second. The more modern 3080 Ti has 912 gibabytes per second.
Let that sink in: ONE TERABYTE PER SECOND!
They really are that fast.
My old work machine has a GTX 980, and even that has 225 GB/s of bandwidth. It's a 7 year old card! Let's use that to make things fair and realistic.
I have a 4K screen plugged into it, so the frame buffer is 31 MB. That works out to 7,282 frames per second, or 137 microseconds per frame.
The GPU hardware is not the issue here. It's entirely possible to do it in microseconds even on very old hardware.
I find that the question of "whether simpler" is summed up in the aphorism "it's easy to get wrong answers infinitely fast".
Where software gets out of control really quickly is wherever you start to express a thing that is not essentially true on its own premises, and requires the introduction of a great number of assumptions. Commercial software falls into this pit unthinkingly since the business requirements create all kinds of assumptions: "it needs to support as many languages as possible" or "it needs to have calendar features" or "it needs to be compatible with this other system" are three super common ones that can create huge dependencies. So our software deviates from "truth" in some respect to get the job done of being presentable to other humans and doing the thing they expect.
At every layer of our current systems, we have little Swiss Army Knife components that can be set up to do anything to resolve these kinds of issues, which means that they do all of it somewhat poorly. Old software is often more like a chainsaw - it doesn't have much to configure, so it's inadmissable for a larger spectrum of tasks, but the thing it does, it does quickly.
Long run, though, I believe that true software crowds out false, even if there's a short-term advantage in getting the market to adopt something that does everything poorly and mostly serves the business, the next phase of that is a flight to quality, which currently manifests as everyone depending on some unmaintained open source code that works and does the job.
Love this.
Doesn’t Microsoft terminal have to be backwards compatible for a ton of historic features though? I would imagine that those sorts of requirements would essentially hobble it. Breaking backwards compatibility isn’t too intriguing for their bottom-line.
It seems like microsoft should have just said that fixing this isn’t a priority right now, instead of saying it couldn’t be done.
It's great though when someone says it's not possible to make something faster, and you say “here, I made it” and leaving the existing console to scroll while you do the rest of the video is great showmanship.
I interviewed Casey[1] and I learned a lot from his perspective on development, particularly not being afraid to just build things yourself from scratch and to try to understand what is going on at the level of hardware.
> It seems like microsoft should have just said that fixing this isn’t a priority right now, instead of saying it couldn’t be done.
The problem is that if performance isn't a priority, and then you finish all of your features and ship your 1.0, it begs the question: when exactly are you going to make it fast?
Never. That's when. Once you've shipped something, people get assigned to other projects, or new features.
Worse, you're now locked in to bad architectures that too much code depends on for changes to be easily made. Casey makes the point that the Windows Console IO API is inherently bad, and using it guarantees an unfixable bottleneck.
I've seen this attitude in Microsoft's software everywhere I look. They simply flat out ignore performance as a feature, and almost never come back to fix it. Easily less than 1% of the time, no matter how important it is. Even when people make a lot of public noise about some performance issue (like in this case), it quickly turns out that it is now unfixable because of the endemic apathy towards optimisation throughout that organisation.
E.g.: Windows Updates. I have gigabit internet and a high-end SSD, yet it takes something like an hour to apply patches! https://borncity.com/win/2018/06/01/windows-server-2016-slow... That to me is simply unfathomably bad programming. How hard is it to download a file and copy it to a folder?
The simpler code here would be orders of magnitude faster than whatever craziness they're doing. Like Casey points out, I could sit down right now and write a Windows Update that runs a thousand times faster. It wouldn't take much more than a weekend. (I'm not kidding: Prior to Windows Server 2019, all versions of Windows Update had O(n^3) algorithms operating on hundreds of megabytes of XML data that could take nearly a day to process.)
Remember the Grand Theft Auto game loading performance issue that was posted here where some guy cut minutes out of the game loading time by binary-patching it? People here were estimating that Rockstar may have missed out on a billion dollars of revenue because huge numbers of players gave up in frustration and moved on to other games: https://news.ycombinator.com/item?id=26296339
Microsoft had their own version of this. When they released Flight Simulator 2020 a lot of people called it "download simulator" because it would take a solid day to download and install. Many people made excuses for it, but they shouldn't have. Just by inspecting the progress bar I found at least five critical performance mistakes. Again: I have gigabit internet and an SSD, yet that install took 8 hours!
12 comments
[ 2.8 ms ] story [ 39.0 ms ] threadI had one where tuning the code to fit the cache size (by using a simpler, smaller, "less efficient" algorithm) led to a ludicrous performance gain ... and several burnt 85Mhz Sparc CPUS, they expected pauses occasionally on the data bus apparently.
Most of the time the optimizations work as advertised but there is no problem contriving examples that show ridiculous improvements by removing speed optimization.
There are various limits that have huge effect on your code performance. For example, if your inner application loop does not fit instruction cache its performance might be impaired. If you can reduce the size of the code, ironically by removing speed optimizations, this might just be what is needed to dramatically speed it up.
On macro scale years ago I have been using Gentoo and, generally, compiling almost everything with size optimization had very positive performance improvement on a platform that has been deficient in RAM.
I guess "size" improvement should not be thought as opposite of "speed" improvement but rather an alternate way to possibly improve performance. The compiler does not know which alternative may or may not bring effects.
Note that this whole twitter thread is in reference to Casey lodging an issue on GitHub about how slow Windows Terminal is, wherein he was told that his simple suggestion for making it fast would be a PhD level thesis worth of work. He was being polite and explaining things to them until then, but after that statement he was so disgusted he closed the issue and created refterm to prove his concept[0]. Then a bunch of people came out of the woodwork to make excuses for why it still couldn't be done. It's a fucking terminal emulator, a piece of 80s technology[1] that mimics a 60s technology, running in 2021, and people still think it makes any sense at all that it could be slow!?
[0] which is not anything groundbreaking, it's pretty much the obvious way to do things. He has intentionally done no optimization since it's designed to serve as a minimum performance benchmark.
[1] I'm guessing here, since I can't readily find an answer to when the first terminal emulator was released.
All the rest of the time is bookkeeping and memory transfers.
GPUs are FAST, but frame data is BIG.
My older RTX 2080 Ti has main memory bandwidth (not on-chip cache bandwidth!) of 616 gigabytes per second. The more modern 3080 Ti has 912 gibabytes per second.
Let that sink in: ONE TERABYTE PER SECOND!
They really are that fast.
My old work machine has a GTX 980, and even that has 225 GB/s of bandwidth. It's a 7 year old card! Let's use that to make things fair and realistic.
I have a 4K screen plugged into it, so the frame buffer is 31 MB. That works out to 7,282 frames per second, or 137 microseconds per frame.
The GPU hardware is not the issue here. It's entirely possible to do it in microseconds even on very old hardware.
Where software gets out of control really quickly is wherever you start to express a thing that is not essentially true on its own premises, and requires the introduction of a great number of assumptions. Commercial software falls into this pit unthinkingly since the business requirements create all kinds of assumptions: "it needs to support as many languages as possible" or "it needs to have calendar features" or "it needs to be compatible with this other system" are three super common ones that can create huge dependencies. So our software deviates from "truth" in some respect to get the job done of being presentable to other humans and doing the thing they expect.
At every layer of our current systems, we have little Swiss Army Knife components that can be set up to do anything to resolve these kinds of issues, which means that they do all of it somewhat poorly. Old software is often more like a chainsaw - it doesn't have much to configure, so it's inadmissable for a larger spectrum of tasks, but the thing it does, it does quickly.
Long run, though, I believe that true software crowds out false, even if there's a short-term advantage in getting the market to adopt something that does everything poorly and mostly serves the business, the next phase of that is a flight to quality, which currently manifests as everyone depending on some unmaintained open source code that works and does the job.
It's great though when someone says it's not possible to make something faster, and you say “here, I made it” and leaving the existing console to scroll while you do the rest of the video is great showmanship.
I interviewed Casey[1] and I learned a lot from his perspective on development, particularly not being afraid to just build things yourself from scratch and to try to understand what is going on at the level of hardware.
[1] https://corecursive.com/062-game-programming/
The problem is that if performance isn't a priority, and then you finish all of your features and ship your 1.0, it begs the question: when exactly are you going to make it fast?
Never. That's when. Once you've shipped something, people get assigned to other projects, or new features.
Worse, you're now locked in to bad architectures that too much code depends on for changes to be easily made. Casey makes the point that the Windows Console IO API is inherently bad, and using it guarantees an unfixable bottleneck.
I've seen this attitude in Microsoft's software everywhere I look. They simply flat out ignore performance as a feature, and almost never come back to fix it. Easily less than 1% of the time, no matter how important it is. Even when people make a lot of public noise about some performance issue (like in this case), it quickly turns out that it is now unfixable because of the endemic apathy towards optimisation throughout that organisation.
E.g.: Windows Updates. I have gigabit internet and a high-end SSD, yet it takes something like an hour to apply patches! https://borncity.com/win/2018/06/01/windows-server-2016-slow... That to me is simply unfathomably bad programming. How hard is it to download a file and copy it to a folder?
The simpler code here would be orders of magnitude faster than whatever craziness they're doing. Like Casey points out, I could sit down right now and write a Windows Update that runs a thousand times faster. It wouldn't take much more than a weekend. (I'm not kidding: Prior to Windows Server 2019, all versions of Windows Update had O(n^3) algorithms operating on hundreds of megabytes of XML data that could take nearly a day to process.)
Remember the Grand Theft Auto game loading performance issue that was posted here where some guy cut minutes out of the game loading time by binary-patching it? People here were estimating that Rockstar may have missed out on a billion dollars of revenue because huge numbers of players gave up in frustration and moved on to other games: https://news.ycombinator.com/item?id=26296339
Microsoft had their own version of this. When they released Flight Simulator 2020 a lot of people called it "download simulator" because it would take a solid day to download and install. Many people made excuses for it, but they shouldn't have. Just by inspecting the progress bar I found at least five critical performance mistakes. Again: I have gigabit internet and an SSD, yet that install took 8 hours!
https://github.com/linpengcheng/PurefunctionPipelineDataflow