> But in general, SPIR-V, as a standard, has two quite different “profiles” to accommodate the needs of graphics like Vulkan and compute like OpenCL. These two “profiles” correspond to the capabilities in the Shader and Kernel tree respectively. They have quite different approaches towards memory models, runtime shader/kernel ABI etc. For the Shader side, it’s using logical addressing mode where we have abstract pointers (no physical size, cannot store into memory by default, etc.) and generally rely on indices to access nested structures. There are quite a lot of requirements on how resources are represented (e.g., using structs with special decorations) and how they match across shader stages (vertex/geometry/fragment/etc.). For the Kernel side, it’s using physical addressing mode, where it’s more akin to LLVM’s approach towards pointers and you can do all sorts of casting. Resources and interfaces are also simpler, without the need to match multiple shader stages and interact with fixed-function stages. So to me, among the two, the Kernel side is more of a fit to go through LLVM flow. Letting the Shader side go through LLVM you might need to fight hard with LLVM transformations assuming more of the Kernel mentality, e.g., recovering structured control flow, handling convergence, threading decorations needed through various layers, etc. Some of the issues have been discussed for a long time in the community and I’m not sure we have a good answer towards them even today.
Note that part of the thread specifically deals with the issues of SPIR-V being split into two different profiles (Shader for graphics and Kernel for compute APIs).
The Shader dialect, which is used for OpenGL 4.6 and Vulkan, is not exactly especially practical for general GPGPU use.
It's the Achilles' heel of Vulkan compute and significantly affects this effort too.
I am a complete noob regarding these subjects so pardon my ignorance but does it mean they intent to compile C++ to SPIR-V (or something that's GPUs can understand)? Does it mean short circuiting CUDA?
Compiling C++ to the SPIR-V dialect used by Vulkan in a usable form is close to technical impossibility...
To the SPIR-V dialect used by OpenCL and oneAPI Level Zero is a production-ready exercise heavily used by Intel's oneAPI compute stack. However, note that neither NVIDIA nor AMD support SPIR-V in their OpenCL implementation.
Isn't the main thing missing pointer support in shaders? As I understand it the VK_KHR_buffer_device_address is mandatory in 1.3 and should fix most of the blockers
CUDA is the Nvidia GPU compute infrastructure and runtime. The native language is a variant of C++, but there are lots of ways to use CUDA from Python (very popular in the machine learning world), Julia, etc. It is an extremely mature and polished way to get access to high performance compute. The biggest downside is that it's Nvidia proprietary.
OpenCL is the older non-Nvidia standard infrastructure for GPU compute. It's languishing though. It is traditionally programmed in OpenCL C, but it does support other languages such as SYCL (another C++ dialect, similar in scope to CUDA C++).
HLSL is similar in that it's a language for programming GPU code, also derived from C (with some C++ sprinkled in), but it's different than CUDA and OpenCL in that it's primarily for graphics applications rather than general purpose compute. The line blurs a bit with the concept of "compute shader," which can be seen as a cut-down CUDA or OpenCL that runs inside the GPU graphics API. Compute shaders are becoming more powerful in each generation. That said, HLSL is overwhelmingly popular for game development, because the tools are so good.
This stuff is confusing. I haven't even touched on the different intermediate representations and the tools for converting between the different languages, of which there is an increasingly dense matrix.
Thank you. Could you please also elaborate on this part.
> Additionally DXC can be used to generate SPIR-V which can then be used either with the Vulkan runtime or, through SPIR-V cross, converted to GLSL or Metal Shader Language for use with their respective APIs.
Is it like saying HSLS can be converted to GSLS or Metal Shaders by converting it to some intermediate representation first?
Exactly that. You compile down to SPIR-V, then use spirv-cross to generate Metal Shading Language or GLSL. It's clunky and there are some rough edges - these conversions aren't 100% lossless - but it does work. If you play a game on a mac, there's a very good chance this is the compilation strategy used.
It may be close to a technical impossibility, but the Circle compiler by Sean Baxter is attempting it. That's based on an aggressive "de-pointerization" (see [1] in particular for details). There's also academic work[2] to compile C++ to shaders. I agree that it's an open question how well that will work out.
Also as pointed out elsethread, now that buffer device address is starting to land, the friction to compile pointer-intense C++ code should decrease even more. These are exciting times!
With the Physical Address Buffer extension, many of the pain points in this area aren't impossible anymore, even if Vulkan SPIR-V overall is still painfull vs CUDA/HIP/ROCM.
Would love to see a GCC backend for these types of things although can't help but think the way GCC works wrt to cross compiling will always hold it back.
If we assume the first paragraph is true then only the second dominates the ability of the GCC developers to work on this kind of thing.
LLVM allows you to build one compiler with N backends, whereas GCC basically doesn't. This is very annoying if you want to compile a mixture of code to (say) native object code and shaders.
LLVM's design is fundamentally superior in this context - LLVM can "fork" itself inside an LLVM compiler whereas the way GCC is designed basically means all the hooks don't really like having multiple backends connected to the same frontend. It's probably possible, but it's something where GCC's baggage does actually hold it back even if the basic algorithms are exactly the same (and better implemented sometimes, GCC is still king across the whole benchmark suite even if LLVM is increasingly good on a few).
GCC forks, I am aware are nothing new, but the future of programming is not multiple monolithic compilers but rather (say) C++ (or D, e.g. DCompute would be a real hack if implemented on GCC) with shader and WASM bytecode output as part of one compiler.
GCC can and does do the stuff above but it's just a bit hacky and "please scroll through our mailing list" whereas LLVM is much more welcoming and is a proper library to be consumed.
The only thing LLVM has over GCC is the license for companies that don't like sharing.
I can enjoy most of C++20 on GCC, whereas good luck waiting for clang to reach parity.
GCC has more frontends and backends available in total than LLVM has yet to achieve, despite being "better".
One of the reasons for Rust-GCC existence is exactly the lack of support for the backends that Linux kernel doesn't want to compromise only to allow Rust into the kernel.
LLVM is a better codebase than GCC despite all that. There is a real technical reason to not go with GCC as I have elaborated on above, the GCC codebase really does have its warts even if it's remarkably clear considering how old it is in places. Just read the code. It's obvious that LLVM has been structured around being consumed as a library, whereas GCC was famously not designed to do that - e.g. LOTS of preprocessor magic versus templates, it's a bit 1980s even if they have done a superb job cleaning it up.
I am a GCC fanboy too, but just be realistic.
Also GCC really needs a kick up the arse when it comes to onboarding new contributors and CI etc. They don't need to go full webscale but I think there's a real risk that GCC ends up being maintained by an increasingly small group of people who eventually just die/retire-off.
The way GCC is tested is quite dumb too, things are pulled but then weeks later someone finds out it failed on such and such a a target because they happened to run the testsuite. It's not terrible but I can't imagine it encourages new contributors.
The bitcode used for watchOS is not stable. There’s a special ABI for watchOS that uses 32-bit pointers on 64-bit hardware to make the IR representation stable, but the bitcode itself is still version locked to the compiler.
Apple’s bitcode submission tooling relies on bitcode being forward upgradable, and properties of the ABI to allow it to be retargeted to different SoCs.
> DXIL, the intermediate representation used for DirectX 12, is based on LLVM-3.7 IR encoded as bitcode. For many reasons that have been discussed at length within the LLVM community in the past, this is not great, but it has already shipped and is the driver interface for all DirectX 12 GPU drivers.
I assume they can't upgrade to more recent LLVM versions, as the 3.7 bitcode is now found in a ton of published games. If they move to, say, 3.8, they need a LLVM 3.7 to 3.8 bitcode translator.
Basically, microsoft published someone else's unstable, unsupported API.
On most 3D APIs, with exception of OpenGL (and its Web variants), shaders are shipped as binaries, so if the driver expects another version that game won't play.
I can think of a few off the top of my head (it ties you to an old llvm version) but at the same time, presumably you need to reduce the size on the wire of this format so you need some kind of bitcode, and since it seems to be a binary interface for GPU drivers which can't be changing regularly the choice seems as good as any other?
Note: these are all naive uneducated assumptions on my part, as I'm not well informed in this area.
LLVM IR is a complex, unstable, implementation-defined internal format used by LLVM with lots of quirks. While LLVM can consume LLVM bitcode made by previous versions of LLVM, it must be quite painful to use for any non-LLVM-based tooling. It's also not a portable format: the IR encodes many target-specific assumptions.
It's great to see stacks that forked off from earlier versions of LLVM coming back to the main tree.
The SPIRV backend is mentioned in the discussion, and I'm hopeful that they'll find a way for all the components to compose well together at their various levels of the architecture.
What I remain curious about, is whether NVIDIA will someday follow suite. IIRC CUDA is based on an oldish LLVM version. While there's an AMDGPU target that's actively developed, and a way for one to build CUDA with a recent LLVM version, it'd be great to have everything close to trunk.
Using LLVM to target GPUs is one of those ideas that's a lot prettier on paper than it is in practice. There are so many problems… nonetheless, multiple vendors invested a lot in this approach and are stuck with it.
At least trying to integrate DirectX and HLSL upstream might force LLVM to fix some of the GPU pain points?
The fact there is no upstream SPIR-V backend after all these years doesn't give me much hope though.
35 comments
[ 3.6 ms ] story [ 44.9 ms ] threadNote that part of the thread specifically deals with the issues of SPIR-V being split into two different profiles (Shader for graphics and Kernel for compute APIs).
The Shader dialect, which is used for OpenGL 4.6 and Vulkan, is not exactly especially practical for general GPGPU use.
It's the Achilles' heel of Vulkan compute and significantly affects this effort too.
To the SPIR-V dialect used by OpenCL and oneAPI Level Zero is a production-ready exercise heavily used by Intel's oneAPI compute stack. However, note that neither NVIDIA nor AMD support SPIR-V in their OpenCL implementation.
(and indeed it helps significantly)
CUDA is the Nvidia GPU compute infrastructure and runtime. The native language is a variant of C++, but there are lots of ways to use CUDA from Python (very popular in the machine learning world), Julia, etc. It is an extremely mature and polished way to get access to high performance compute. The biggest downside is that it's Nvidia proprietary.
OpenCL is the older non-Nvidia standard infrastructure for GPU compute. It's languishing though. It is traditionally programmed in OpenCL C, but it does support other languages such as SYCL (another C++ dialect, similar in scope to CUDA C++).
HLSL is similar in that it's a language for programming GPU code, also derived from C (with some C++ sprinkled in), but it's different than CUDA and OpenCL in that it's primarily for graphics applications rather than general purpose compute. The line blurs a bit with the concept of "compute shader," which can be seen as a cut-down CUDA or OpenCL that runs inside the GPU graphics API. Compute shaders are becoming more powerful in each generation. That said, HLSL is overwhelmingly popular for game development, because the tools are so good.
This stuff is confusing. I haven't even touched on the different intermediate representations and the tools for converting between the different languages, of which there is an increasingly dense matrix.
> Additionally DXC can be used to generate SPIR-V which can then be used either with the Vulkan runtime or, through SPIR-V cross, converted to GLSL or Metal Shader Language for use with their respective APIs.
Is it like saying HSLS can be converted to GSLS or Metal Shaders by converting it to some intermediate representation first?
Also as pointed out elsethread, now that buffer device address is starting to land, the friction to compile pointer-intense C++ code should decrease even more. These are exciting times!
[1]: https://github.com/seanbaxter/shaders#approaching-circle-sha...
[2]: https://arxiv.org/abs/2109.14682
They would never used GCC regardless of how GIMPLE and backend infrastructure might be.
LLVM allows you to build one compiler with N backends, whereas GCC basically doesn't. This is very annoying if you want to compile a mixture of code to (say) native object code and shaders.
The first is definitely an issue, how many Nintendo or Sony contributions have landed on upstream?
Or the interesting detail that while standard LLVM bitcode isn't stable, the one used by watchOS is.
Finally the increasing pain that despite the amount of LLVM users topping Linux kernel like usage, clang is now lagging behind C++ compliance.
Which is one of the reasons why GNU C Compiler got renamed as GNU Compiler Collection.
LLVM isn't even anything new as idea.
At very least PL.8, Amsterdam Compiler Toolkit, TrenDRA, MSR Phoenix exist as prior art.
LLVM's design is fundamentally superior in this context - LLVM can "fork" itself inside an LLVM compiler whereas the way GCC is designed basically means all the hooks don't really like having multiple backends connected to the same frontend. It's probably possible, but it's something where GCC's baggage does actually hold it back even if the basic algorithms are exactly the same (and better implemented sometimes, GCC is still king across the whole benchmark suite even if LLVM is increasingly good on a few).
GCC forks, I am aware are nothing new, but the future of programming is not multiple monolithic compilers but rather (say) C++ (or D, e.g. DCompute would be a real hack if implemented on GCC) with shader and WASM bytecode output as part of one compiler.
GCC can and does do the stuff above but it's just a bit hacky and "please scroll through our mailing list" whereas LLVM is much more welcoming and is a proper library to be consumed.
I can enjoy most of C++20 on GCC, whereas good luck waiting for clang to reach parity.
GCC has more frontends and backends available in total than LLVM has yet to achieve, despite being "better".
One of the reasons for Rust-GCC existence is exactly the lack of support for the backends that Linux kernel doesn't want to compromise only to allow Rust into the kernel.
I am a GCC fanboy too, but just be realistic.
Also GCC really needs a kick up the arse when it comes to onboarding new contributors and CI etc. They don't need to go full webscale but I think there's a real risk that GCC ends up being maintained by an increasingly small group of people who eventually just die/retire-off.
The way GCC is tested is quite dumb too, things are pulled but then weeks later someone finds out it failed on such and such a a target because they happened to run the testsuite. It's not terrible but I can't imagine it encourages new contributors.
GCC is doing quite alright, looking at upstream contributions for C++, Fortran and Ada compliance.
LLVM on the other hand, I guess those lechers are quite happy not having to upstream anything.
Apple’s bitcode submission tooling relies on bitcode being forward upgradable, and properties of the ABI to allow it to be retargeted to different SoCs.
ouch!
Basically, microsoft published someone else's unstable, unsupported API.
LLVM IR provides upwards compatibility, that’s not what the issue is.
The issue is rather having your game binaries not work at all on older OS/driver releases when built with a newer LLVM version.
I can think of a few off the top of my head (it ties you to an old llvm version) but at the same time, presumably you need to reduce the size on the wire of this format so you need some kind of bitcode, and since it seems to be a binary interface for GPU drivers which can't be changing regularly the choice seems as good as any other?
Note: these are all naive uneducated assumptions on my part, as I'm not well informed in this area.
What I remain curious about, is whether NVIDIA will someday follow suite. IIRC CUDA is based on an oldish LLVM version. While there's an AMDGPU target that's actively developed, and a way for one to build CUDA with a recent LLVM version, it'd be great to have everything close to trunk.
At least trying to integrate DirectX and HLSL upstream might force LLVM to fix some of the GPU pain points?
The fact there is no upstream SPIR-V backend after all these years doesn't give me much hope though.