I'm looking forward to trying these when they stabilize! I currently use WGPU for graphics, and cudarc for CUDA.
Note: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.
I haven't tried the tile API yet; looking forward to it.
The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.
cudarc been great for me, because it's easy to look up existing examples and references, and it maps 1-to-1 with what I see. I'm already having a tough time with CUDA itself, a dialect of it makes a tad harder to rely on previous work.
Seems more ergonomic in general though, both approaches they share, compared to cudarc, and less build infrastructure and fiddling with environments, which is great.
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.
Nobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.
kernels aren't meant to be written by any defined language. C is just a traditionally good default language that took over from assembly. No particular reason we have to stick with C.
It gets fun when someone uses an uncensored model to bypass a refusal, but they accidentally use one that was trained for erotic writing and brings that talent to the documentation task.
McDonald's food is not even that unhealthy. I just tried a Burger King burger the other day and it's terrible. I think it's like 2000 calories in a single burger or something.
I get the impression that Nvidia employees don't care too much - I started seeing fully AI-written "documentation" on some of their smaller projects more than a year ago (i.e., before it was even slightly a good idea).
I hadn't read the article and read this comment as though NVIDIA themselves were implying that this library was checked but not trusted by them since it was fully LLM generated.
Dude I am in slop fucking hell right now. There is still room for a human touch, without which the agents will lever us harder and faster into a world of incomprehensible garbage.
First of all, this is a pre-1.0 release that requires a nightly Rust compiler (if you choose the SIMT track with cuda-oxide) so that one is going to be unstable software.
Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).
Another dependency into the list and course everything is open source except CUDA itself.
In this age of LLM written everything which has softly killed my motivation for learning Rust somewhat, this has revived my interest if not only for the fact the LLMs haven't yet been trained on this yet!
I think OP’s point is that the payoff in learning a new language has diminished in this AI era. You can call that lazy, I’d consider it smart to consider whether you could be doing other, better, things with your time.
what this article tells me is that no one at Nvidia actually cares about this project whatsoever. otherwise, they would have had a person actually write the announcement.
I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both.
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc.
These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
> Once you have allowed that proprietary cr*p into your C++ codebase
People have been doing that all the time for every kind of codebase. It's just a matter of doing business. I don't see how it's worth having any emotions or opinions about it.
Are win32 APIs proprietary? So you decide to use them, or use some wrapper, or don't develop for Windows. Easy choice.
Developing for embedded devices? So you read the manufacturers manual and implement based on the spec, or use some sort of HAL if they are available, or you don't have a job. Even simpler.
> The CUDA runtime is a special case of one of the libraries provided by the CUDA Toolkit. The CUDA runtime provides both an API and some language extensions to handle common tasks such as allocating memory, copying data between GPUs and other GPUs or CPUs, and launching kernels. The API components of the CUDA runtime are referred to as the CUDA runtime API.
> I strongly dislike CUDA. Once you have allowed that proprietary cr*p
Genuine question...why not just type "crap"? It's not even that much of a curse, but I've never really understood the point of self-censorship. If you don't want to curse then you could just use a non-curse word.
It may be to bypass censorship, rather than self-censorship. Some platforms block or shadowban comments with curse words. Not sure about this platform.
Launching kernels manually is an error prone PITA which I believe is the principle reason for CUDA's popularity. Having the compiler give an error when you mess up is a huge benefit. But having the compiler allow you to express "I want to launch this kernel over a grid with these dimensions, with these arguments" as a single expression is where the vast majority of the value comes from.
The having it all in a single file is mostly an artefact of the fact that it is C++, because C++ is single file at a time compilation. In D (which is multiple files in a single compiler invocation) with DCompute (which targets CUDA and OpenCL with upcoming support for Vulkan and Metal), you are required to write the kernels in a separate module, but you get all the benefits of the compiler complaining when you mess up _and_ the expressivity of "launch me this kernel".
62 comments
[ 3.1 ms ] story [ 67.7 ms ] threadNote: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.
I haven't tried the tile API yet; looking forward to it.
The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.
Seems more ergonomic in general though, both approaches they share, compared to cudarc, and less build infrastructure and fiddling with environments, which is great.
[1] https://github.com/huggingface/candle
Damn even Nvidia is putting out fully Claude-written articles.
Why "even Nvidia"?
They are fully behind using AI for basically everything.
What's next? "Damn, even McDonald's is putting out unhealthy food"
They are just better at hiding it or configuring Claude.
I have several skills that reformat text to remove AI-speak tells.
Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).
Another dependency into the list and course everything is open source except CUDA itself.
There were humans far superior than you for writting Rust before LLM, now there's a LLM. The only difference is price and time execution.
You get an awesome teacher (LLM) ready to answer all your questions about Rust.
And you still find excuses not to learn it ?
At some point, just realize you've been lazy to learn it and LLMs are just an excuse.
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc. These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
People have been doing that all the time for every kind of codebase. It's just a matter of doing business. I don't see how it's worth having any emotions or opinions about it.
Are win32 APIs proprietary? So you decide to use them, or use some wrapper, or don't develop for Windows. Easy choice.
Developing for embedded devices? So you read the manufacturers manual and implement based on the spec, or use some sort of HAL if they are available, or you don't have a job. Even simpler.
From: https://docs.nvidia.com/cuda/cuda-programming-guide/01-intro...
Genuine question...why not just type "crap"? It's not even that much of a curse, but I've never really understood the point of self-censorship. If you don't want to curse then you could just use a non-curse word.
The having it all in a single file is mostly an artefact of the fact that it is C++, because C++ is single file at a time compilation. In D (which is multiple files in a single compiler invocation) with DCompute (which targets CUDA and OpenCL with upcoming support for Vulkan and Metal), you are required to write the kernels in a separate module, but you get all the benefits of the compiler complaining when you mess up _and_ the expressivity of "launch me this kernel".
This is more promising: https://github.com/Rust-GPU/rust-gpu/