That certainly convinced me. When I was doing my taxes recently and had to watch those forced loading animations, I kept asking myself "why can't my compiler do this?" Thanks to std::simd, now it can!
This is the first time I've seen a classic for loop called a "boomer loop", but apparently this isn't even the first instance (not the first definition):
I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate non-trivial SIMD code. Neither can the compiler. Portability just amplifies this gap.
I think a legitimate criticism is that it is unclear who std::simd is for. People that don’t use SIMD today are unlikely to use std::simd tomorrow. At the same time, this does nothing for people that use SIMD for serious work. Who is expected to use this?
The intrinsics are not difficult but you do have to learn how the hardware works. This is true even if you are using a library. A good software engineer should have a rough understanding of this regardless.
> I think a legitimate criticism is that it is unclear who std::simd is for. People that don’t use SIMD today are unlikely to use std::simd tomorrow. At the same time, this does nothing for people that use SIMD for serious work. Who is expected to use this?
There is plenty of vectorization that are simple enough to be done with std::simd today and that will still bring any autovectorizer begging on its knees for various reasons.
As an anecdote, I currently got a 8x speedup with std::simd (AVX2 & SVE2) on a rather trivial parser of mine recently that autovectorizer failed miserably to do properly.
Would I have get better result using intrinsics ? Likely, yes.
Did I want to suffer the maintainability and portability pain associated with it for a simple parser ? Certainly not.
For these use case, std.simd does the job.
And will probably do a better and wider job with time when it get enriched by the committee.
The blog brings some valid criticism but really looks like a flame war trying to destroy an already opened door.
(1) Is there more performant solutions that std::simd for vectorization ?
Yes, of course.
The STL evolves slow, its main goal is to provide a generic and portable implementations of a set of algorithms. Not to provide the best implementation in existence.
The best implementation of most algorithms (including SIMD patterns) evolves every 6 month, you can not expect a standard library with 3 different implementation to keep up with that.
(2) Is the future of vectorization ISPC ?
Nope. ISPC has been around for > 10y and is still niche.
There is very good reasons to that: Yes it can generate better code but in most use case, adding a massive dependency of a compiler + an arbitrary LLVM version + a DSL on your project is not worth it.
Specially considering that it is an Intel project and that Intel (almost) abandonned the project multiple time (In pure Intel fashion).
So yes, criticism is easy, and yes std::simd is full of problems.
But I am glad it exists, and thanks to the people that made it happen... Because it is useful, even in the current state.
> If I want good results, I have to write intrinsics.
Any good book / article / tutorial to begin learning it? Most docs I've tried assume you already know what you can do and focus on how to express it in assembly or with intrinsics. However, I'm not even aware of what operations are commonly implemented in hardware. For example, FMA is not normally tought, is not expressed in higher-level languages... how am I supposed to imagine such a thing exists, even before looking it up in a reference manual?
I made the first proposal to the C++ standard committee to introduce SIMD in 2011, before Matthias Kretz got involved with his own version (which is what became std::simd). This was based on what eventually became Eve (mentioned in the article).
Back then, it was rejected, for the same arguments that people are making today, such as not mapping to SVE well, having a separate way to express control flow etc.
There was a real alternative being considered at the time: integrating ISPC-like semantics natively in the language. Then that died out (I'm not sure why), and SIMD became trendy, so the committee was more open to doing something to show that they were keeping up with the times.
GCC already solved it:
https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html
The operations behave like C++ valarrays. Addition is defined as the addition of the corresponding elements of the operands. For example, in the code below, each of the 4 elements in a is added to the corresponding 4 elements in b and the resulting vector is stored in c.
Maybe there's an interesting story in there, it's certainly possible. But the "author" could not be bothered to write it, and so why should we waster our time reading it?
If you thought std::simd was a library nobody asked for, just wait until you hear about <linalg>. I feel like half the people looking forward to that think they're just going to get standard C++ bindings to LAPACK, when instead they're probably going to get an unoptimized, slapdash implementation of LAPACK written by people who aren't good at BLAS.
As for SIMD itself, designing a good SIMD library is difficult because there are several different SIMD approaches and some of them work poorly for certain use cases. For example, you can take an HPC-ish approach of "vectorize this loop" (à la #pragma omp simd) and have the compiler take care of a fairly mechanical transformation. Or you can take an opposite approach of treating a 128-bit SIMD vector as a fundamental data type in your language. Which approach is better depends on your use case.
> The problem is that std::simd in 2026 is the 2012 solution arriving after the world moved on. The committee spent a decade polishing a library-based approach while compilers solved the easy cases automatically and ISPC solved the hard cases with language-level support.
I find it interesting that the C++ committee would make that kind of mistake. Shouldn't they know better?
C++ sits on that weird abstraction level where it wants to be a higher level language but it keeps grinding their gears on stuff like pointer sizes, pointer arithmetic or vector sizes and at the same time wants to keep being C compatible and needs that interface with the lower level world
Now compare with how numpy does things: you care about the data size but not the implementation.
Still, I didn't expect less (of a crap fest) from the C++ committee as presented here
Just write inline asm for x86 and aarch64 (if you care about that) and not care about the rest. Is it even useful to do simd on other processors?
Compiler optimizing even the code around the simd code based on the semantics of arithmetic or other things sounds silly after writing some of this kind of code
Currently experimental, but looks like the first Intel arch will arrive in the next release in about 3 months. They are also going to support a portable layer.
Wondering what people here think about the approach the Go team is taking; I think they would appreciate more eyeballs on their design. (I’m not competent in this space (yet))…
I think there will be a "portable" wrapper that will also support other architectures (arm, riscv, even wasm) in the future based on the Highway C++ library.
I'd actually rather just have the compiler give some guarantees on producing SIMD code when you write regular C++ code doing sums, multiplications, etc... in a particular way. And perhaps add a few more operators/keywords to the language for modern CPU instructions (we got things like popcount, countl_zero and fma, but what about e.g. pext, pdep, aes, ...)
The point about the optimizer only seeing "opaque templates and function calls" makes little sense.
First off, templates are the opposite of opaque due to the fundamental requirement that the implementation be visible to every translation unit using a template. This makes any function calls trivially inlinable.
Second, and the reason for the above requirement, templates are compiled by monomorphization – making a distinct, separately optimizable copy of each concrete instantiation of a template. By the time the compiler backend sees the intermediate representation, there’s nothing about templates left.
There are of course reasons why highly abstracted template code may be difficult to optimize, for instance if function call chains are so deep that the inliner gives up. There are also legitimate reasons why a fully language-based solution might beat a library-based one. But one of the points of adding a library to the std is that the standard library is allowed to cheat as much as it wants. It can be deeply integrated to the compiler and implemented entirely using compiler magic if necessary.
std::simd may be too little, too late for many reasons, but I doubt any of them is that the compiler can’t see through the code.
Agreed, fixed with vectors needs to be a language feature, better compile times and would solve issues for most people.
Personally, I think that like Clang way to adding GLSL like vectors and semantics would've gone a long way. SVE might be an elegant design, but in reality there are probably a multiple factor of game and other 3d code being written that needs vectors compared to other fields, and there limited vector sizes aren't really a problem.
And honestly, considering the story of AVX512.. with 512 bit vectors being removed from mainstream by Intel, do we really really need longer ones despite it being from a "scalable design"?
Something everyone is missing is that this is just feature parity with other langauges, this is not c++ specific. They added it because other languages have these compiler hints which are then (usually) used in llvm as opaque types for 'smarter' optimizations. Hand rolled code will still be better, but there are very niche instances where you want llvm to know this information, especially important when you don't care about performance, but you care about data integrity and obfuscation.
> The pattern is clear: every major project that actually needs portable SIMD in production chose a third-party library or a different language
of course they chose third-party library, because C++26 is just only published and don't have wide support/adoption experience.
> And the most damning data point might be EVE itself — a committee member looked at std::simd, decided it wasn’t good enough, and built his own library.
It's just a manipulation. First commit in eve[1] was published in 2018. There was no any std::simd in standard at that time.
> Nobody waited for std::simd. By the time it ships in C++26, these libraries will have a decade of production battle-testing, real user feedback, and cross-platform coverage that std::simd can’t match on day one.
Manipulation 2. No library have decade "of production battle-testing, real user feedback, and cross-platform coverage" on day one. So, why their authors created them?
> Including <experimental/simd> pulls in deeply nested template machinery — simd.h, simd_x86.h, simd_builtin.h, and friends. A trivial function computing sin on a SIMD vector takes about 2.2 seconds to compile. The equivalent scalar for-loop? 0.2 seconds.
Would be more interesting if you compare this with precompiled headers and C++20 modules.
> The std::simd version? It emits actual vsqrtps + vmulps because the optimizer can’t perform algebraic simplification through opaque template function calls:
opaque template function calls? What is this?
Of course there is 1000 examples when compiler can do better job with scalar loop. And there is 1000 examples when it can't. But, for some reason people do write simd manually. Probably because they want predictable code generation - no massive slowdown on another compiler/another compiler version/another cpu/another one line of loop changed.
> sqrt(x) * sqrt(x)
what compiler would generate with manual simd intrinsincs? I doubt the same as scalar mul.
> The frustrating part is that the problems are well-understood. SIMD programmers have been asking for the same things for years, and none of them are in std::simd.
Show me your proposal with critique of std::simd, if you asking for them. Or at least someone other proposal.
How you can understand that someone asking?
Too many emotional statements in, too little technical details.
> When Google needed portable SIMD for production image and video codecs, they built Highway — not std::simd.
Sure, they left the committee years ago. I am not trying to claim any sort of direct causality, but it sure seems like this is a case where Google's presence on the committee might have prevented shipping boondoggles like this. Modules is another case where I think Google's feedback might have been able to steer the ship in a better direction.
"The Default Width Problem" -- this section seems confused and definitely reeks of LLM authorship. It's comparing -march=native against std::simd and complaining that std::simd<T,8> breaks portability with pre-Haswell. This is a real issue, but -march=native is no better! It bakes in the SIMD width at compile-time as well, so that binary also won't run on a pre-Haswell machine. It's a real issue but neither side solves it. You need runtime dispatch (a la Google Highway) to solve this.
If we want to improve cross-platform SIMD, in my opinion we should start by supporting more operations in LLVM IR. Like vector expansion (currently we only have expandload), runtime-known shuffle vectors, pdep/pext operations.
Also, let's stop with the "vector length agnostic" types being the sole option for SVE extensions. I'd rather write an optimized routine for a 16-byte machine I'm targeting and be able to upgrade it in 5 years than have "agnostic" code that wants to pretend like it would work amazingly on all platforms, but the machine I optimized it for is theoretical. I'm fine with recompiling my code, I do it every day. If I have an algorithm that's truly vector length agnostic, I can make the vector length a constant in my code that can change based on the compile target.
One thing I will point out is that the code in the article is compiled with `-march=native` and `-ffast-math`, meaning that they're really only compiling for the exact same machine they are running on and no other. This seems like it is mainly applicable to places which can easily recompile code for the exact known hardware that they run on, such as HFT and some scientific computing.
Places which compile code to distribute for people to run on a variety of processors and platforms (or that require floating point code to be consistent between them), i.e. games and applications, will still be targetting a low end baseline architecture and therefore have a different outcome. I can say that in this space we are only now reaching the point where we can start compiling for AVX2, as we can expect the lowest end-user processor to support it.
32 comments
[ 4.4 ms ] story [ 61.5 ms ] thread[1]: https://github.com/NoNaeAbC/std_simd
http://boomer-loop.urbanup.com/18229646
I think a legitimate criticism is that it is unclear who std::simd is for. People that don’t use SIMD today are unlikely to use std::simd tomorrow. At the same time, this does nothing for people that use SIMD for serious work. Who is expected to use this?
The intrinsics are not difficult but you do have to learn how the hardware works. This is true even if you are using a library. A good software engineer should have a rough understanding of this regardless.
There is plenty of vectorization that are simple enough to be done with std::simd today and that will still bring any autovectorizer begging on its knees for various reasons.
As an anecdote, I currently got a 8x speedup with std::simd (AVX2 & SVE2) on a rather trivial parser of mine recently that autovectorizer failed miserably to do properly.
Would I have get better result using intrinsics ? Likely, yes.
Did I want to suffer the maintainability and portability pain associated with it for a simple parser ? Certainly not.
For these use case, std.simd does the job. And will probably do a better and wider job with time when it get enriched by the committee.
The blog brings some valid criticism but really looks like a flame war trying to destroy an already opened door.
(1) Is there more performant solutions that std::simd for vectorization ?
Yes, of course. The STL evolves slow, its main goal is to provide a generic and portable implementations of a set of algorithms. Not to provide the best implementation in existence.
The best implementation of most algorithms (including SIMD patterns) evolves every 6 month, you can not expect a standard library with 3 different implementation to keep up with that.
(2) Is the future of vectorization ISPC ?
Nope. ISPC has been around for > 10y and is still niche. There is very good reasons to that: Yes it can generate better code but in most use case, adding a massive dependency of a compiler + an arbitrary LLVM version + a DSL on your project is not worth it.
Specially considering that it is an Intel project and that Intel (almost) abandonned the project multiple time (In pure Intel fashion).
So yes, criticism is easy, and yes std::simd is full of problems.
But I am glad it exists, and thanks to the people that made it happen... Because it is useful, even in the current state.
Any good book / article / tutorial to begin learning it? Most docs I've tried assume you already know what you can do and focus on how to express it in assembly or with intrinsics. However, I'm not even aware of what operations are commonly implemented in hardware. For example, FMA is not normally tought, is not expressed in higher-level languages... how am I supposed to imagine such a thing exists, even before looking it up in a reference manual?
Back then, it was rejected, for the same arguments that people are making today, such as not mapping to SVE well, having a separate way to express control flow etc.
There was a real alternative being considered at the time: integrating ISPC-like semantics natively in the language. Then that died out (I'm not sure why), and SIMD became trendy, so the committee was more open to doing something to show that they were keeping up with the times.
Maybe there's an interesting story in there, it's certainly possible. But the "author" could not be bothered to write it, and so why should we waster our time reading it?
As for SIMD itself, designing a good SIMD library is difficult because there are several different SIMD approaches and some of them work poorly for certain use cases. For example, you can take an HPC-ish approach of "vectorize this loop" (à la #pragma omp simd) and have the compiler take care of a fairly mechanical transformation. Or you can take an opposite approach of treating a 128-bit SIMD vector as a fundamental data type in your language. Which approach is better depends on your use case.
You optimize for a specific target.
The problem is that you cannot be cross-platform. Sure.
But that is why software is incremental.
I write for my HW, not yours. You can write for yours.
Make folders with implemntations
x86_v1 x86_v2 arm64 riscv64 ... ... ...
and include
> The problem is that std::simd in 2026 is the 2012 solution arriving after the world moved on. The committee spent a decade polishing a library-based approach while compilers solved the easy cases automatically and ISPC solved the hard cases with language-level support.
I find it interesting that the C++ committee would make that kind of mistake. Shouldn't they know better?
C++ sits on that weird abstraction level where it wants to be a higher level language but it keeps grinding their gears on stuff like pointer sizes, pointer arithmetic or vector sizes and at the same time wants to keep being C compatible and needs that interface with the lower level world
Now compare with how numpy does things: you care about the data size but not the implementation.
Still, I didn't expect less (of a crap fest) from the C++ committee as presented here
Compiler optimizing even the code around the simd code based on the semantics of arithmetic or other things sounds silly after writing some of this kind of code
Currently experimental, but looks like the first Intel arch will arrive in the next release in about 3 months. They are also going to support a portable layer.
Wondering what people here think about the approach the Go team is taking; I think they would appreciate more eyeballs on their design. (I’m not competent in this space (yet))…
I think there will be a "portable" wrapper that will also support other architectures (arm, riscv, even wasm) in the future based on the Highway C++ library.
https://godbolt.org/z/n8hKhc7rY
(click the recompile button if you don't see the Asm code)
First off, templates are the opposite of opaque due to the fundamental requirement that the implementation be visible to every translation unit using a template. This makes any function calls trivially inlinable.
Second, and the reason for the above requirement, templates are compiled by monomorphization – making a distinct, separately optimizable copy of each concrete instantiation of a template. By the time the compiler backend sees the intermediate representation, there’s nothing about templates left.
There are of course reasons why highly abstracted template code may be difficult to optimize, for instance if function call chains are so deep that the inliner gives up. There are also legitimate reasons why a fully language-based solution might beat a library-based one. But one of the points of adding a library to the std is that the standard library is allowed to cheat as much as it wants. It can be deeply integrated to the compiler and implemented entirely using compiler magic if necessary.
std::simd may be too little, too late for many reasons, but I doubt any of them is that the compiler can’t see through the code.
Personally, I think that like Clang way to adding GLSL like vectors and semantics would've gone a long way. SVE might be an elegant design, but in reality there are probably a multiple factor of game and other 3d code being written that needs vectors compared to other fields, and there limited vector sizes aren't really a problem.
And honestly, considering the story of AVX512.. with 512 bit vectors being removed from mainstream by Intel, do we really really need longer ones despite it being from a "scalable design"?
of course they chose third-party library, because C++26 is just only published and don't have wide support/adoption experience.
> And the most damning data point might be EVE itself — a committee member looked at std::simd, decided it wasn’t good enough, and built his own library.
It's just a manipulation. First commit in eve[1] was published in 2018. There was no any std::simd in standard at that time.
> Nobody waited for std::simd. By the time it ships in C++26, these libraries will have a decade of production battle-testing, real user feedback, and cross-platform coverage that std::simd can’t match on day one.
Manipulation 2. No library have decade "of production battle-testing, real user feedback, and cross-platform coverage" on day one. So, why their authors created them?
> Including <experimental/simd> pulls in deeply nested template machinery — simd.h, simd_x86.h, simd_builtin.h, and friends. A trivial function computing sin on a SIMD vector takes about 2.2 seconds to compile. The equivalent scalar for-loop? 0.2 seconds.
Would be more interesting if you compare this with precompiled headers and C++20 modules.
> The std::simd version? It emits actual vsqrtps + vmulps because the optimizer can’t perform algebraic simplification through opaque template function calls:
opaque template function calls? What is this?
Of course there is 1000 examples when compiler can do better job with scalar loop. And there is 1000 examples when it can't. But, for some reason people do write simd manually. Probably because they want predictable code generation - no massive slowdown on another compiler/another compiler version/another cpu/another one line of loop changed.
> sqrt(x) * sqrt(x)
what compiler would generate with manual simd intrinsincs? I doubt the same as scalar mul.
> The frustrating part is that the problems are well-understood. SIMD programmers have been asking for the same things for years, and none of them are in std::simd.
Show me your proposal with critique of std::simd, if you asking for them. Or at least someone other proposal. How you can understand that someone asking?
Too many emotional statements in, too little technical details.
[1] https://github.com/jfalcou/eve
Sure, they left the committee years ago. I am not trying to claim any sort of direct causality, but it sure seems like this is a case where Google's presence on the committee might have prevented shipping boondoggles like this. Modules is another case where I think Google's feedback might have been able to steer the ship in a better direction.
Also, let's stop with the "vector length agnostic" types being the sole option for SVE extensions. I'd rather write an optimized routine for a 16-byte machine I'm targeting and be able to upgrade it in 5 years than have "agnostic" code that wants to pretend like it would work amazingly on all platforms, but the machine I optimized it for is theoretical. I'm fine with recompiling my code, I do it every day. If I have an algorithm that's truly vector length agnostic, I can make the vector length a constant in my code that can change based on the compile target.
https://github.com/llvm/llvm-project/issues/113422
https://github.com/llvm/llvm-project/issues/172857
Places which compile code to distribute for people to run on a variety of processors and platforms (or that require floating point code to be consistent between them), i.e. games and applications, will still be targetting a low end baseline architecture and therefore have a different outcome. I can say that in this space we are only now reaching the point where we can start compiling for AVX2, as we can expect the lowest end-user processor to support it.