Extension in this context basically just means new registers/instructions that didn't exist on a 386. Even floating point support on x86 is technically an extension. SSE (which adds the xmm registers) has existed on all x86 CPUs made by Intel in the last 20+ years, and the ymm registers/instructions are part of AVX2 which has existed on all x86 CPUs made by Intel in the last 10+ years.
I don't mean mmx through avx2 when I said extensions. I mean the "vector length" extensions/flag for avx512 that lets you know you can use those instructions with xmm and ymm sized operands. Since it is a flag it means it is a theoretical optional feature whereas it should have just been part of the avx512 base. With the hindsight of what Intel want for avx10 there should have been flags for xmm, ymm, and zmm operands. That would be excellent for the "E" cores.
"Intel AVX10 Version 1 will be introduced for early software enablement and supports the subset of all the Intel AVX-512 instruction set available as of future Intel Xeon processors with P-cores, codenamed Granite Rapids, that is
forward compatible to Intel AVX10. This version will not include the new 256-bit vector instructions supporting embedded rounding or any of the new instructions and will serve as the transition base version from Intel AVX-512 to Intel AVX10.
Intel AVX10 Version 2 will include the 256-bit instruction forms supporting embedded rounding as well as a suite of new Intel AVX10 instructions covering new AI data types and conversions, data movement optimizations, and standards support. All new instructions will be supported at 128-, 256-, and 512-bit vector lengths with limited variances. All Intel AVX10 versions will implement the new versioning enumeration scheme."
And who knows when AMD will have time to update Zen ? architecture with these new instructions.
The latest shipping processor is the 4-th generation Intel Xeon "Sapphire Rapids". AVX10 looks like it won't be fully supported until the 6-th generation "Granite Rapids" Xeons, probably in 2025.
So AMD has plenty of time to implement the additional instructions, of which there don't appear to be that many. Mostly it's a unification of feature sets and a simplification of the CPU ID flags.
Also, Intel is still nerfing the efficiency "E" cores to AVX10-256, so the instruction sets will remain non-uniform in practice for the foreseeable future.
Meanwhile, AMD is implementing AVX-512 on all of their CPU cores including in the high-density 128-core AMD EPYC "Bergamo" line, which is a significant differentiator.
Intel just cannot resist segmenting a market, even when announcing that they're ending market segmentation!
> So AMD has plenty of time to implement the additional instructions
Architectures take years to plan. If AMD is just finding out, I'm not sure they have time.
However, one would hope AMD had a little heads up. Its in Intel's and AMD's best interest not to flub the next AVX, with ARM SVE2 (and who knows what else) around the corner.
Hasn't AMD proven multiple times that a double pumped packed-SIMD implementation works well enough? Just the permute operations need a full width data path to get reasonable latencies. Intel already overplayed their hand with AVX-512 when they still had a stronger position. Let's hope they fail to hold back the field with their misguided attempts to increase their margin no matter the cost (even to their own bottom line).
My understanding is that, while double pumped 512 bit makes sense from AMD from a compatibility perspective, it's not a great solution. The sheer amount of architectural stare you need to keep track of with all these 512 bit instructions gets unwieldy, especially on smaller cores, and there's no real performance advantage to double pumped 512 vs normal 256.
This new ISA seems to include 256 bit versions of the instructions which were previously only in AVX-512 though. That's exciting.
Actually on Zen 4 using 512-bit instructions instead of 256-bit instructions frequently improves the performance a lot, despite using the same execution units.
The reason is that in Zen 4 the relatively narrow instruction decoder is frequently the bottleneck, so twice less instructions can influence the speed very much.
On future CPUs that will be able to decode more instructions in parallel, like Zen 5 and the future Intel CPUs, 512-bit instructions might no longer increase the speed when the width of the execution units remains of 256 bits, so perhaps even AMD may introduce a split between laptop CPUs that implement only the 256-bit subset of AVX-512 rebranded AVX10 and server CPUs that implement the full AVX-512 a.k.a. AVX10.
The architectural state is quadrupled, but the real impact is much smaller.
Single-pumped AVX2 implementations already had ~150 extra 256-bit registers for pipelining purposes. And if you're double pumping 512-bit instructions, then you can pipeline just as deep without changing the quantity or width.
Maintaining similar performance is not 512B->2KB, it's 5KB->7KB.
(But note that the E cores are largely double-pumping 256 bit instructions already, so for them it's a big change.)
Is someone here who understands the nitty bitty details of AVX-512/AVX10 and could tell me what is included which current latest gen AMD processors do not support?
Because the only thing I can pick out is the 256bit AVX-512 which AFIK recent amd processors do support (including 512bit support) both on their normal cores and their new compacted code.
But I don't know much about AVX_ so I'm 100% I missed a bunch of stuff and/or limitations with current AMD code (besides it being double pumped).
> Is someone here who understands the nitty bitty details of AVX-512/AVX10 and could tell me what is included which current latest gen AMD processors do not support?
Zen 4 lacks AVX512_FP16 (for 16-bits IEEE floating point operations), AVX512_VP2INTERSECT and also lack the Advanced Matrix eXtension (AMX) set (if you consider that part of AVX512).
You can see a table here showing cpu's AVX512 support[0], and wikicpu also has that data in a list [1], along with more detailed info. It's worth noting all data I can find specifically talks about AVX512 support in terms of the architecture (Zen4), not anything specific product families such as ryzen vs epyc implementations. It's possible there is/will be some artificial product segmentation going on (like what Intel did retroactively with Alder Lake to disable AVX512) but I've not seen any mention of it regarding AMD's products. I mention this because Intel has kinda been all over the place with AVX512 implementation, and there's a big difference between Enterprise and Consumer in how they deliver/treat AVX512. AMD obviously is a different company with different practices, but with only 1 generation to go off of it's hard to speculate on a trend, and AVX512 can be an expensive implementation. Point being: It's unfortunately complicated AND subject to change.
The "big" one here for ML/DL/AI vs Intel's Sapphire Rapids (and presumably going forward) is FP16 not being supported on AMD. Of course, it's contextual as to whether that's relevant to you though.
My experience is primarily with Intel's implementation of AVX512 in their Xeon products for DL/AI. I can expand on that if interested, but unfortunately haven't had a chance to play with AMD Zen4 yet in that domain, and it's gonna be a while till I can get hands on with a Epyc 9000 probably.
deleted my initial reply as I misread "bf16" in your comment as fp16 and was thoroughly confused. What I Get for commenting before fully awake, sorry lol.
And yes, true. That list I just copied from wikichip, didn't think to cross verify what it was conveying in those paranthesis as the question was about AMD not Intel. That said, not sure why they have FP16 attributed to Cooper Lake and not SR.
EDIT: Ayy, you can edit wikichip. Fixed it over there (assuming there's not some moderation issue? never submitted anything over there), but too late to edit original HN Comment.
One of the reasons to use Intel is that their L1 cache is sized to 512-bit operations.
AMD Zen4 has 256-bit load/store to L1 cache, which means it'd take a load/store unit 2-clock ticks to execute any 512-bit load or store. (Though Zen4 has multiple load/store units, so maybe its not that big of a deal in practice).
-------------
GPUs are 32-wide with 32-bits each, or 1024-wide operations in practice. GCN was 64-wide with 32-bits each or 2048 wide in practice.
The reason why AVX512 was a big deal isn't the width btw. But instead the design of the instruction set. AVX512 is the most advanced CPU vector instruction set ever made, with huge inspirations from NVidia's PTX or AMD's GCN.
Of course, GPU-assembly remains superior. I don't think that RISC-V (or whoever) should be copying AVX512 or even AVX10, they should be looking at NVidia PTX or NVidia SASS as inspiration.
1. AVX512 allows a 64-bit conditional mask to be used over the 64-bytes (aka a 512-bit operation). Like GPUs (NVidia or AMD) before it, this allows for far simpler implementations of if-statements (or loops) in practice.
2. AVX512 has vgather and vscatter, aka vector-load and vector-store. While no computer accelerates this process, this still simplifies programming and read/writes to memory can be thought of on a per-lane basis.
3. Compress and Expand, new instructions not found in GPU-space, perform the stream-compression operation (and inverse operation). See: https://www.cse.chalmers.se/~uffe/streamcompaction.pdf , lots of practical applications here, nice to see just a simple "instruction" that handles this detail. Though GPU programmers can prefix-sum and build this primitive themselves in practice, its a common enough operation that a proper-instruction is likely a good idea.
------------
Just a few tidbits of AVX512 ISA that shows that Intel really knows what they're doing here.
#1 can be seen as a 64-bit generalization to the cmov conditional instructions that are in Intel's normal 64-bit instruction set. Except instead of being a "1-bit" compare or 1-bit if-statement, might as well generalize it to a 64-bit operation across 64x8-bit bytes that naturally map into a 512-bit register, right?
Its "why" you can just say "if(blahblah)" in CUDA or OpenCL, because under-the-hood, the GPUs are holding onto these 64-bit conditional registers and are turning-off and turning-on lanes to execute. (GPUs don't do if(){A();} else () {B();} like a CPU. GPUs turn on conditional-execution masks per-lane and then execute both A-and-B, though if the lane is "off" then the results are thrown away). It all comes down to S-registers (GCN), opcode-masks (AVX512) or whatnot, its fundamental to "proper" GPU-like SIMD.
In any case, AVX512 is still playing catchup to the gpu space, but Intel has shown me (IMO) that they're the CPU-assembly group that has fully understood what makes GPUs tick and why parallelism is possible in CUDA / OpenCL. Maybe they'll leapfrog the GPU guys in the next few years, but I still consider GPUs to be the cream-of-the-crop in terms of assembly-language design for parallel SIMD systems... for today anyway.
Then again, NVidia has also shown themselves to be very innovative in this low-level space. And in many ways, its easier to "catch up" (as Intel has done) rather than trailblaze. Still, Intel showing off compression instructions is a good sign, it is true innovation.
RVV has masking for everything (masks being stored as packed bits in a vector register), memory gather & scatter, compress (& easily emulatable expand via an instruction that expands 0,1,2,3,… by which you can then shuffle), and in addition to that is scalable (i.e. same code can run on hardware with anything between 128-bit and 65536-bit vector registers; plus something alike "hardware unrolling" where operations can be configured to run over groups of 1/2/4/8 registers; and a vector length option in addition to masking allowing trivially handling the tail of loops with the same body as the main loop, without affecting masking), and is still more orthogonal than AVX-512 (e.g. AVX-512 still only has 8- & 16-bit saturating add/sub, missing 8-bit shifts, whereas RVV has orthogonal everything (even high half of 64-bit product, allowing for two-instruction 64×64→128-bit multiplication)). Granted, it's significantly newer than AVX-512.
This seemed really cool. I'm used to a lot of new instructions & boosts, but Intel adding new conditional load/store is a smart interesting coupling that could help increase execution unit efficiency in a significant way.
> As out-of-order CPUs continue to become deeper and wider, the cost of mispredictions increasingly dominates performance of such workloads. Branch predictor improvements can mitigate this to a limited extent only as data-dependent branches are fundamentally hard to predict.
> To address this growing performance issue, we significantly expand the conditional instruction set of x86, which was first introduced with the Intel® Pentium® Pro in the form of CMOV/SET instructions. These instructions are used quite extensively by today’s compilers, but they are too limited for broader use of if-conversion (a compiler optimization that replaces branches with conditional instructions).
> Intel® APX adds conditional forms of load, store, and compare/test instructions, and it also adds an option for the compiler to suppress the status flags writes of common instructions.
I didn't understand everything about the "caller-saved volatile" new general purpose register interface & legacy compatibility. But some potentially really interesting optimizations where load/store being dual register capable, and being capable of staying on the AVX unit & not having to go further out to "memory" (caches?):
> Generally, more register state will need to be managed at function boundaries. In order to reduce the associated overhead, we are adding PUSH2/POP2 instructions that transfer two register values within a single memory operation. The processor tracks these new instructions internally and fast-forwards register data between matching PUSH2 and POP2 instructions without going through memory.
> I didn't understand everything about the "caller-saved volatile"
Whenever new registers are added to a CPU, it may be necessary to modify the operating system in order to save the new registers across context switches, but it is possible to change the ABI (i.e. the conventions for calling functions) in such a way that will allow the new functions that use the new registers to be linked with old functions that are not aware of the existence of the new registers.
To achieve this goal, it is enough to define in the ABI all the new registers as "caller-saved volatile".
This means that any function that uses some of the new registers must not assume anything about the values of those registers upon function entry and it must save them before calling any other function.
This ensures that regardless what other functions do, intentionally or accidentally, the values stored in the new registers cannot be corrupted, so it is safe to mix new functions that are aware of them with old functions that ignore them.
Had the new registers been defined e.g. as registers that must be preserved across calls, i.e. which must be saved inside the invoked function, that would not have worked, because an old function would not have known how to save them.
Not every function boundary is an ABI boundary, and compilers can and will automatically use custom ABIs for static functions (and, with LTO, for unexported global functions, too).
Why did it take around 10 years for AMD to implement AVX-512 and will they need to wait as long for this too? Doesn't seem to be patent related (patents are 20 years and AVX-512 was introduced in 2013?).
Because Intel implementations of AVX-512 were pretty much unusable or not worth using until 2019-2020, so nobody was writing much AVX-512, so they didn't need to.
AMD practically resurrected AVX-512 by supporting it across both desktop and server products, so it's weird to criticize them for taking a while when Intel nearly killed the thing via segmentation and poor implementation.
AVX512 vector and mask register take space on the die, which is a finite resource and could be allocated to other things.
Moreover, power consumption may have also been a challenge, as seen in the early Intel implementations of AVX512 (especially Skylake-X).
And while AVX512 can improve single-threaded performance, that's only for applications that make use of it. And up until recently, that number was extremely limited.
I'm guessing the reason AVX512 took so long is that doubling the size increases complexity by a lot. This doesn't introduce wider vectors, so I don't see why it would take so long.
But who knows if AMD sees value in these new instructions/if there's market demand for them.
As an average developer who works on high level interface, I don't really see the benefit of AVX-512. I've heard that some math calculating software MAY gains some benefit from AVX instructions (like BLAS), but I've never use it personally. Can you guys please explain?
Effectively, they let you do this in chunks of 2, 4 or 8 elements in a single instruction instead of element by element. Usually the clock speed of the processor drops slightly while executing these instructions but not so much as to make it slower than not having the instruction.
New instruction sets usually do two things - add support for new operations, and widen the registers and allow you to perform the operation on more elements at a time.
If you’re writing in a high level language, you’ll only really see performance improvements if your interpreter or library takes advantage of them. For e.g. the Python library NumPy makes good use of vectorisation.
In terms of writing low level code, simple loops are often auto-vectorised by the compiler so you often see a sort of odd style where a loop doing many things is split up so the compiler can deal with it. You often end up having to run the code through something like VTune to work out whether a particular loop has actually vectorised.
Anything which works on nontrivial datasets, really. As soon as you need to do roughly the same operation on a bunch of data, you can benefit from AVX. Heck, this could be as simply as determining the length of a string!
The main benefit of AVX-512 is that the CPU gained support for "masking". Traditionally, you had to execute the instructions on all data elements. Something like "round all even values downwards, round all odd values upwards" becomes near-impossible, and if you want to do that in an entire AVX pipeline you have to deconstruct the vector into individual elements, do the operation on each of them individually, and reconstruct it into a vector. This really sucks. With masking, you get a special field which specify on which elements the operation should apply. So you could just create a mask with the even values, use that mask for a round-down, invert the mask, and use it for a round-up. That's a significant speedup!
Adding AVX-512 support means that a lot more applications suddenly become eligible for fairly trivial vectorization, and I personally can't wait for it to become universally available.
47 comments
[ 3.3 ms ] story [ 53.5 ms ] thread> being able to work for both P and E cores
Oh yes I forgot they were gimping their own processors.
> the converged version has a maximum vector length of 256-bits [on] the E cores while P cores will have optional 512-bit vector use
Maybe they shouldn't have made xmm and ymm "extensions" to the base set to begin with.
"Intel AVX10 Version 1 will be introduced for early software enablement and supports the subset of all the Intel AVX-512 instruction set available as of future Intel Xeon processors with P-cores, codenamed Granite Rapids, that is forward compatible to Intel AVX10. This version will not include the new 256-bit vector instructions supporting embedded rounding or any of the new instructions and will serve as the transition base version from Intel AVX-512 to Intel AVX10.
Intel AVX10 Version 2 will include the 256-bit instruction forms supporting embedded rounding as well as a suite of new Intel AVX10 instructions covering new AI data types and conversions, data movement optimizations, and standards support. All new instructions will be supported at 128-, 256-, and 512-bit vector lengths with limited variances. All Intel AVX10 versions will implement the new versioning enumeration scheme."
And who knows when AMD will have time to update Zen ? architecture with these new instructions.
So AMD has plenty of time to implement the additional instructions, of which there don't appear to be that many. Mostly it's a unification of feature sets and a simplification of the CPU ID flags.
Also, Intel is still nerfing the efficiency "E" cores to AVX10-256, so the instruction sets will remain non-uniform in practice for the foreseeable future.
Meanwhile, AMD is implementing AVX-512 on all of their CPU cores including in the high-density 128-core AMD EPYC "Bergamo" line, which is a significant differentiator.
Intel just cannot resist segmenting a market, even when announcing that they're ending market segmentation!
Architectures take years to plan. If AMD is just finding out, I'm not sure they have time.
However, one would hope AMD had a little heads up. Its in Intel's and AMD's best interest not to flub the next AVX, with ARM SVE2 (and who knows what else) around the corner.
Guess Intel’s feeling the pressure from Zen 4 supporting AVX-512.
This new ISA seems to include 256 bit versions of the instructions which were previously only in AVX-512 though. That's exciting.
The reason is that in Zen 4 the relatively narrow instruction decoder is frequently the bottleneck, so twice less instructions can influence the speed very much.
On future CPUs that will be able to decode more instructions in parallel, like Zen 5 and the future Intel CPUs, 512-bit instructions might no longer increase the speed when the width of the execution units remains of 256 bits, so perhaps even AMD may introduce a split between laptop CPUs that implement only the 256-bit subset of AVX-512 rebranded AVX10 and server CPUs that implement the full AVX-512 a.k.a. AVX10.
Single-pumped AVX2 implementations already had ~150 extra 256-bit registers for pipelining purposes. And if you're double pumping 512-bit instructions, then you can pipeline just as deep without changing the quantity or width.
Maintaining similar performance is not 512B->2KB, it's 5KB->7KB.
(But note that the E cores are largely double-pumping 256 bit instructions already, so for them it's a big change.)
Because the only thing I can pick out is the 256bit AVX-512 which AFIK recent amd processors do support (including 512bit support) both on their normal cores and their new compacted code.
But I don't know much about AVX_ so I'm 100% I missed a bunch of stuff and/or limitations with current AMD code (besides it being double pumped).
Half-precision floating-point for sure.
Sapphire Rapids has added an alternative format FP16, which is not supported by Zen 4, because Zen 4 was launched before Sapphire Rapids.
https://twitter.com/InstLatX64/status/1646471371558461445/
It's worth noting that 16-bits floating point operations are still possible on Zen 4, but using Google's BF16.
- AVX512ERAVX512PF (Knights Landing)
- AVX512 4VNNIW, 4FMAPS (Knights Mill)
- VP2INTERSECT (Tiger Lake)
- FP16 (Cooper Lake)
You can see a table here showing cpu's AVX512 support[0], and wikicpu also has that data in a list [1], along with more detailed info. It's worth noting all data I can find specifically talks about AVX512 support in terms of the architecture (Zen4), not anything specific product families such as ryzen vs epyc implementations. It's possible there is/will be some artificial product segmentation going on (like what Intel did retroactively with Alder Lake to disable AVX512) but I've not seen any mention of it regarding AMD's products. I mention this because Intel has kinda been all over the place with AVX512 implementation, and there's a big difference between Enterprise and Consumer in how they deliver/treat AVX512. AMD obviously is a different company with different practices, but with only 1 generation to go off of it's hard to speculate on a trend, and AVX512 can be an expensive implementation. Point being: It's unfortunately complicated AND subject to change.
The "big" one here for ML/DL/AI vs Intel's Sapphire Rapids (and presumably going forward) is FP16 not being supported on AMD. Of course, it's contextual as to whether that's relevant to you though.
[0]: https://en.wikipedia.org/wiki/AVX-512?oldformat=true#CPUs_wi...
[1]: https://en.wikichip.org/wiki/amd/microarchitectures/zen_4#Ne...
My experience is primarily with Intel's implementation of AVX512 in their Xeon products for DL/AI. I can expand on that if interested, but unfortunately haven't had a chance to play with AMD Zen4 yet in that domain, and it's gonna be a while till I can get hands on with a Epyc 9000 probably.
EDIT: Fixed typos
And yes, true. That list I just copied from wikichip, didn't think to cross verify what it was conveying in those paranthesis as the question was about AMD not Intel. That said, not sure why they have FP16 attributed to Cooper Lake and not SR.
EDIT: Ayy, you can edit wikichip. Fixed it over there (assuming there's not some moderation issue? never submitted anything over there), but too late to edit original HN Comment.
AMD Zen4 has 256-bit load/store to L1 cache, which means it'd take a load/store unit 2-clock ticks to execute any 512-bit load or store. (Though Zen4 has multiple load/store units, so maybe its not that big of a deal in practice).
-------------
GPUs are 32-wide with 32-bits each, or 1024-wide operations in practice. GCN was 64-wide with 32-bits each or 2048 wide in practice.
The reason why AVX512 was a big deal isn't the width btw. But instead the design of the instruction set. AVX512 is the most advanced CPU vector instruction set ever made, with huge inspirations from NVidia's PTX or AMD's GCN.
Of course, GPU-assembly remains superior. I don't think that RISC-V (or whoever) should be copying AVX512 or even AVX10, they should be looking at NVidia PTX or NVidia SASS as inspiration.
Can you explain what makes it the above? I haven't really worked with avx512 before, only up to avx2 and rvv.
2. AVX512 has vgather and vscatter, aka vector-load and vector-store. While no computer accelerates this process, this still simplifies programming and read/writes to memory can be thought of on a per-lane basis.
3. Compress and Expand, new instructions not found in GPU-space, perform the stream-compression operation (and inverse operation). See: https://www.cse.chalmers.se/~uffe/streamcompaction.pdf , lots of practical applications here, nice to see just a simple "instruction" that handles this detail. Though GPU programmers can prefix-sum and build this primitive themselves in practice, its a common enough operation that a proper-instruction is likely a good idea.
------------
Just a few tidbits of AVX512 ISA that shows that Intel really knows what they're doing here.
#1 can be seen as a 64-bit generalization to the cmov conditional instructions that are in Intel's normal 64-bit instruction set. Except instead of being a "1-bit" compare or 1-bit if-statement, might as well generalize it to a 64-bit operation across 64x8-bit bytes that naturally map into a 512-bit register, right?
Its "why" you can just say "if(blahblah)" in CUDA or OpenCL, because under-the-hood, the GPUs are holding onto these 64-bit conditional registers and are turning-off and turning-on lanes to execute. (GPUs don't do if(){A();} else () {B();} like a CPU. GPUs turn on conditional-execution masks per-lane and then execute both A-and-B, though if the lane is "off" then the results are thrown away). It all comes down to S-registers (GCN), opcode-masks (AVX512) or whatnot, its fundamental to "proper" GPU-like SIMD.
In any case, AVX512 is still playing catchup to the gpu space, but Intel has shown me (IMO) that they're the CPU-assembly group that has fully understood what makes GPUs tick and why parallelism is possible in CUDA / OpenCL. Maybe they'll leapfrog the GPU guys in the next few years, but I still consider GPUs to be the cream-of-the-crop in terms of assembly-language design for parallel SIMD systems... for today anyway.
Then again, NVidia has also shown themselves to be very innovative in this low-level space. And in many ways, its easier to "catch up" (as Intel has done) rather than trailblaze. Still, Intel showing off compression instructions is a good sign, it is true innovation.
> As out-of-order CPUs continue to become deeper and wider, the cost of mispredictions increasingly dominates performance of such workloads. Branch predictor improvements can mitigate this to a limited extent only as data-dependent branches are fundamentally hard to predict.
> To address this growing performance issue, we significantly expand the conditional instruction set of x86, which was first introduced with the Intel® Pentium® Pro in the form of CMOV/SET instructions. These instructions are used quite extensively by today’s compilers, but they are too limited for broader use of if-conversion (a compiler optimization that replaces branches with conditional instructions).
> Intel® APX adds conditional forms of load, store, and compare/test instructions, and it also adds an option for the compiler to suppress the status flags writes of common instructions.
https://www.intel.com/content/www/us/en/developer/articles/t...
I didn't understand everything about the "caller-saved volatile" new general purpose register interface & legacy compatibility. But some potentially really interesting optimizations where load/store being dual register capable, and being capable of staying on the AVX unit & not having to go further out to "memory" (caches?):
> Generally, more register state will need to be managed at function boundaries. In order to reduce the associated overhead, we are adding PUSH2/POP2 instructions that transfer two register values within a single memory operation. The processor tracks these new instructions internally and fast-forwards register data between matching PUSH2 and POP2 instructions without going through memory.
Neat stuff. Very superficially reminds me of Semantic Streaming Registers on the very novel standalone-ish FPU on PULP's RISC-V based Occamy many-core chip. In that the unit is acting in a more standalone fashion. https://www.youtube.com/watch?v=kMhdq7A3d3I#t=10m https://pulp-platform.org/docs/BeniniSC11-22.pdf
Whenever new registers are added to a CPU, it may be necessary to modify the operating system in order to save the new registers across context switches, but it is possible to change the ABI (i.e. the conventions for calling functions) in such a way that will allow the new functions that use the new registers to be linked with old functions that are not aware of the existence of the new registers.
To achieve this goal, it is enough to define in the ABI all the new registers as "caller-saved volatile".
This means that any function that uses some of the new registers must not assume anything about the values of those registers upon function entry and it must save them before calling any other function.
This ensures that regardless what other functions do, intentionally or accidentally, the values stored in the new registers cannot be corrupted, so it is safe to mix new functions that are aware of them with old functions that ignore them.
Had the new registers been defined e.g. as registers that must be preserved across calls, i.e. which must be saved inside the invoked function, that would not have worked, because an old function would not have known how to save them.
Not every function boundary is an ABI boundary, and compilers can and will automatically use custom ABIs for static functions (and, with LTO, for unexported global functions, too).
AMD practically resurrected AVX-512 by supporting it across both desktop and server products, so it's weird to criticize them for taking a while when Intel nearly killed the thing via segmentation and poor implementation.
AVX512 vector and mask register take space on the die, which is a finite resource and could be allocated to other things.
Moreover, power consumption may have also been a challenge, as seen in the early Intel implementations of AVX512 (especially Skylake-X).
And while AVX512 can improve single-threaded performance, that's only for applications that make use of it. And up until recently, that number was extremely limited.
But who knows if AMD sees value in these new instructions/if there's market demand for them.
New instruction sets usually do two things - add support for new operations, and widen the registers and allow you to perform the operation on more elements at a time.
If you’re writing in a high level language, you’ll only really see performance improvements if your interpreter or library takes advantage of them. For e.g. the Python library NumPy makes good use of vectorisation.
In terms of writing low level code, simple loops are often auto-vectorised by the compiler so you often see a sort of odd style where a loop doing many things is split up so the compiler can deal with it. You often end up having to run the code through something like VTune to work out whether a particular loop has actually vectorised.
The main benefit of AVX-512 is that the CPU gained support for "masking". Traditionally, you had to execute the instructions on all data elements. Something like "round all even values downwards, round all odd values upwards" becomes near-impossible, and if you want to do that in an entire AVX pipeline you have to deconstruct the vector into individual elements, do the operation on each of them individually, and reconstruct it into a vector. This really sucks. With masking, you get a special field which specify on which elements the operation should apply. So you could just create a mask with the even values, use that mask for a round-down, invert the mask, and use it for a round-up. That's a significant speedup!
Adding AVX-512 support means that a lot more applications suddenly become eligible for fairly trivial vectorization, and I personally can't wait for it to become universally available.