Is there any hope of actually getting the promised performance out of modern hardware without having a CPU/assembly expert on your team?
The situation is ridiculous. I've played games that are simple reimplementations of DOS games (not via an emulator, but using SDL) that use 20% of a modern 2GHz CPU, while the original ran on a 12MhZ machine.
So it sounds like the games were probably doing too much in the re-implementation (60FPS isn't necessary for a DOS game).
To answer the question:
>Is there any hope of actually getting the promised performance out of modern hardware without having a CPU/assembly expert on your team?
The answer is: it's complicated. Let's assume you want absolute maximum performance, so you use a uni-kernel design (quite a few super computers basically do this). Even without the overhead of virtual memory (minimal on modern hardware), the removal of multi-tasking (expensive but mitigatable) on a single core; you would still have to deal with all the complex things you need to do to run modern hardware. Modern hardware is complex, it's complex because we want things to be fast or have high bandwidth (usually both).
Ultimately you can usually get 90% percent of what the hardware can do with a good compiler, decent software architecture, and tuning to encourage vectorization; probably 95% with a great compiler. Even in user mode, even in Windows. So at the end of the day it's about what's more important: Using as few cycles of a CPU, or as few cycles of your time implementing it. I think the re-implementers chose the latter as they chose to let SDL follow its default FPS instead of tuning it downwards.
CPUs are complex. This case is ridiculously visible, but I suspect that only obvious performance-broker pops and are fixed/warned. Non performance-critical "everyday corner cases" fly to production.
I remember a blog post full of very specific cases that compiler don't optimize at all because the case could never appear in a performance sensitive code, but you could meet this situation in many code.
There is a ratio benefit/cost in performance inspection, both human workload and compiler code maintenance/performance. You want to focus on what matter and on big code base where every performance curves are flat, you could think everything has been done.
Also, when I had an AMD Athlon, 2D games up to 1024x768 ran pretty fast. Now it's impossible to keep up with 1280x768 on some 2D games (non-3D) on an AMD Turion.
When you compile stuff in Visual C++ with /arch:AVX or /arch:AVX2 setting, the compiler is using VEX for everything, including SSE instructions which can be encoded the old way. This eliminates the described problem completely. Also eliminates a few others as well, e.g. VEX-encoded unaligned memory access is guaranteed to work, with SSE encoding these same instructions might fail with “unaligned access” runtime exception.
AVX1 and VEX are supported by vast majority of CPUs made after 2011. Steam hardware survey says if you require AVX1, you gonna loose less than 7% of the audience. Maybe it’s time to move on already, and drop support for pre-AVX1 CPUs?
In one of my current projects (started as a green field about 1.5 years ago) the customer was willing to sacrifice compatibility for development cost, I told them I’ll ship sooner if we require AVX2 and D3D feature level 11.0 hardware, they agreed. Over these 1.5 years they only hit that once, trying to run my software on some 12-core Ivy Bridge Xeon, it doesn’t have AVX2 but it does support AVX1 and therefore VEX.
For the enterprise software you need to support old hardware simply because clients (especially Swiss ones) often have a very old VMware environment running on Xeons without AVX. This is either because CPUs do not support AVX or there is a compatibility layer enabled for live migration that disables it.
Probably these Pentiums and Celerons are used by the unfortunate 6.5% of Steam users without AVX1. If I would be working on a casual videogame instead of a resource-intensive CAM/CAE app, I probably wouldn’t want to require AVX.
This is so insane to me. I just don't understand it at all. Pervasive high performance SIMD would have provided a huge moat to porting compute heavy software away from x86.
But since you can't rely on it, nobody wants to maintain a slow path fallback. One that's going to be used on all the slowest machines anyway, so it had better be good too.
By the way, Intel’s SSE (the compatible vector stuff supported by 100% of modern CPUs) and ARM Neon both operate on 128-bit vectors, it’s usually not too hard to port between the two, did it more than once.
AVX code is much harder to port on ARM, as the registers are 256 bits. By disabling AVX in low-end CPUs, Intel is forcing many people to still maintain SSE 4.1 versions. This is making much easier to port software from Intel to ARM.
I think that’s an example of quarterly report-driven management, Intel picked short-term profits at the cost of long-term strategy.
For code using intrinsics (not assembler) https://simd-everywhere.github.io/blog/ might be helpful, including on x86_64, though it doesn't provide runtime dispatch. GCC itself knows a level (sse4.2?) of x86_64 intrinsics on ppc64le.
Yes. Only the paranoid survive era Intel is rolling in its grave.
They used to do everything - legal or otherwise - to stay on top. Now they're not even bothering with the legal stuff.
AMD would have played ball, because it would have benefited them just as much.
Post Sandy Bridge the x86 license was probably proportionally more valuable to them than to Intel. If it went away, Intel still had their fabs, which at that time showed no sign of danger.
If I'm understanding the linked bug report right, this is just plain unpleasant. So, running SSE code mixed with AVX code has a penalty unless you zero out the upper parts of the registers in a special way when switching from AVX to SSE. This seems annoying but no big deal - just save the registers your function needs to preserve using AVX instructions, zero them out, do your SSE computations, and restore them before returning. Not so fast! If the calling code was legacy SSE rather than AVX, this will mark the upper halves of the registers as dirty and incur a performance penalty after the function returns, even though the offending registers have the same contents at the architectural level as they did before the call and the only AVX code is the code intended to clear out and restore the upper halves of the registers in order to prevent exactly this. Apparently, the fix is to test if the upper halves of the registers are zero and use different code paths to save and restore them depending on whether they are. What a mess.
All SSE instructions can be encoded into VEX and won’t trigger the issue. It’s just an unfortunate combination of compiler and linker settings, where compiler used legacy encoding despite AVX was enabled somewhere else.
Modern compilers are aware, e.g. check out gcc 10: https://godbolt.org/z/39v98r I asked it to compile for Sandy Bridge and the compiler has encoded all vector instruction in VEX, you can tell by opcodes starting with v, e.g. vmovmskpd instead of just movmskpd: https://www.felixcloutier.com/x86/movmskpd
That's not much help when the function being called is in a shared system library like ld.so or libc that needs to support being used both by older binaries that use SSE and newer ones that use AVX, which I think is the case here. People aren't just going to recompile every single program on the planet and break backwards compatibility with older systems in the process, so on some level I think there's no alternative but to work around this. (Also, remember that SSE is used for even non-vectorized floating point in modern software, and that a lot of the vectorized stuff is likely to be hand-optimized assembler.)
GNU libc, at least, has versions of a variety of functions for different SIMD variants with runtime dispatch. You can provide that with GCC attributes, or explicitly with the cpuid-like mechanisms typically used in linear algebra libraries, for instance.
22 comments
[ 3.2 ms ] story [ 55.1 ms ] threadThe situation is ridiculous. I've played games that are simple reimplementations of DOS games (not via an emulator, but using SDL) that use 20% of a modern 2GHz CPU, while the original ran on a 12MhZ machine.
To answer the question:
>Is there any hope of actually getting the promised performance out of modern hardware without having a CPU/assembly expert on your team?
The answer is: it's complicated. Let's assume you want absolute maximum performance, so you use a uni-kernel design (quite a few super computers basically do this). Even without the overhead of virtual memory (minimal on modern hardware), the removal of multi-tasking (expensive but mitigatable) on a single core; you would still have to deal with all the complex things you need to do to run modern hardware. Modern hardware is complex, it's complex because we want things to be fast or have high bandwidth (usually both).
Ultimately you can usually get 90% percent of what the hardware can do with a good compiler, decent software architecture, and tuning to encourage vectorization; probably 95% with a great compiler. Even in user mode, even in Windows. So at the end of the day it's about what's more important: Using as few cycles of a CPU, or as few cycles of your time implementing it. I think the re-implementers chose the latter as they chose to let SDL follow its default FPS instead of tuning it downwards.
I remember a blog post full of very specific cases that compiler don't optimize at all because the case could never appear in a performance sensitive code, but you could meet this situation in many code.
There is a ratio benefit/cost in performance inspection, both human workload and compiler code maintenance/performance. You want to focus on what matter and on big code base where every performance curves are flat, you could think everything has been done.
AVX1 and VEX are supported by vast majority of CPUs made after 2011. Steam hardware survey says if you require AVX1, you gonna loose less than 7% of the audience. Maybe it’s time to move on already, and drop support for pre-AVX1 CPUs?
In one of my current projects (started as a green field about 1.5 years ago) the customer was willing to sacrifice compatibility for development cost, I told them I’ll ship sooner if we require AVX2 and D3D feature level 11.0 hardware, they agreed. Over these 1.5 years they only hit that once, trying to run my software on some 12-core Ivy Bridge Xeon, it doesn’t have AVX2 but it does support AVX1 and therefore VEX.
Probably these Pentiums and Celerons are used by the unfortunate 6.5% of Steam users without AVX1. If I would be working on a casual videogame instead of a resource-intensive CAM/CAE app, I probably wouldn’t want to require AVX.
But since you can't rely on it, nobody wants to maintain a slow path fallback. One that's going to be used on all the slowest machines anyway, so it had better be good too.
AVX code is much harder to port on ARM, as the registers are 256 bits. By disabling AVX in low-end CPUs, Intel is forcing many people to still maintain SSE 4.1 versions. This is making much easier to port software from Intel to ARM.
I think that’s an example of quarterly report-driven management, Intel picked short-term profits at the cost of long-term strategy.
They used to do everything - legal or otherwise - to stay on top. Now they're not even bothering with the legal stuff.
AMD would have played ball, because it would have benefited them just as much.
Post Sandy Bridge the x86 license was probably proportionally more valuable to them than to Intel. If it went away, Intel still had their fabs, which at that time showed no sign of danger.
https://ark.intel.com/content/www/us/en/ark/products/199285/...
https://ark.intel.com/content/www/us/en/ark/products/199290/...
AVX2 is supported with i3s:
https://ark.intel.com/content/www/us/en/ark/products/199283/...
Modern compilers are aware, e.g. check out gcc 10: https://godbolt.org/z/39v98r I asked it to compile for Sandy Bridge and the compiler has encoded all vector instruction in VEX, you can tell by opcodes starting with v, e.g. vmovmskpd instead of just movmskpd: https://www.felixcloutier.com/x86/movmskpd