No it doesn’t. AMD drivers don’t support all of the extensions, optimizations, and related in things like automatic1111. There’s always stuff that breaks on AMD and works perfectly in CUDA land.
I do think the original comment was wrong (if they tested in Automatic1111 then ... enough said about it working). However, they still have a point in warning people about AMD cards. There is more risk to these things than Nvidia:
- AMD's support is flaky. The AMD 7900 XTX is only officially supported on Windows PCs (for ROCm). On Linux PCs in my experience there is a high risk of graphics card hard lockups.
- Historically AMD has dropped support for consumer cards really quickly (I don't think they support any consumer cards from more than about 3 years ago). At some point that'll stabilise, but it is hard to tell if this is the generation that will start working long term or the next.
- AMD generally gets second class support in return from the machine learning communities. There still seems to be some chance that you'll be locked out of the latest and greatest if you go AMD.
At some point this will all probably come good and it'll be like CPUs where there is really no difference between the vendors. There is too much interest in machine learning right now for anything else to happen ... but I've been buying AMD for more than a decade now and I've been thinking that for years already. If someone is on the edge they'd be better off not getting seduced by a minor performance improvement and sticking to Nvidia cards. It is uncertain when the long term will reach us. This year? 5 years? A decade? All options where AMDs software drivers are in play.
Although AMDs graphics drivers do seem great on Linux nowadays, I like things that just work. It is a pity they got so out of position vs CUDA.
they don't occur in amdgpu-pro proprietary userland, but AMD apparently runs that on an internal release cycle and doesn't make the current repo or build available to the public. And AMD just historically hasn't cared enough about the open userland to support that even in the officially-supported configurations.
I'll also add to GP by saying that the windows support is all of about 2 weeks old at this point and it's also the first time that consumer cards (as opposed to the CDNA compute cards and the workstation-branded Radeon Pro cards) have had official support. Not that that is a guarantee it'll actually run with AMD though, especially on the open-source userland.
It looks like a kernel panic to me, probably bad kernel drivers but I've never tried to figure out what is happening. My graphics card is more than 3 years old (ie, unsupported) so it isn't worth reporting.
Yeah, if they're going to compare techniques breaking various features and extensions, I think it's only fair to compare it to similarly incompatible techniques for nvidia.
In which case, I would recommend the AITemplate extension[0] for ComfyUI, which runing at a (hopefully) standardized 512x512 res, default euler A sampler, nets me about 23.8 it/s on my 350w 3090.
> AMD drivers don’t support all of the extensions, optimizations, and related in things like automatic1111.
With olive in general, you need to rebuild all the models for it; and a whole lot of the extensions for A1111 are support for workflow components with new models (often, whole new classes of models). Unless Olive becomes a major platform for people developing models, this is always going to be lagging.
Wait, why are they comparing Microsoft Olive on AMD to Pytorch on Nvidia? Nvidia supposedly shipped support for Olive recently, there should be no problem getting a head-to-head comparison: https://www.tomshardware.com/news/nvidia-geforce-driver-prom...
The comments point out that AMD in the table performing well required the use of Microsoft Olive, and someone in the article comments implies that if you use Microsoft Olive with Nvidia instead of Pytorch with Nvidia, then you'll see the Nvidia jump in performance as well, largely rendering the supposed leap by AMD not relevant. Is that true? Can folks chime in?
> Using Microsoft Olive and DirectML instead of the PyTorch pathway results in the AMD 7900 XTX going form a measly 1.87 iterations per second to 18.59 iterations per second!
So the headline should be Microsoft Olive vs. PyTorch and not AMD vs. Nvidia.
The results of the usual benchmarks are inconclusive between the 7900 XTX and the 4080, Nvidia is only somewhat more expensive, yet CUDA is much more popular than anything AMD is allowed to support. So I’d say this makes sense as an AMD vs Nvidia comparison as well.
I’m not sure which customer willing to spend $1000-1200 to do ML workflows isn’t willing to spend $1600 to get another 20%+ of performance and have the fastest card available.
I’m not saying people have unlimited budgets but it just seems like the choice most people in that price range would make.
While hobbyist users are not a significant chunk of the market, you can almost bet that they will want the best bang for buck irrespective of the performance of an individual card.
Tesla P40 is being heavily discussed in these circles because although it's janky (extremely bad TFLOPS for 16-bit operations), it's still thought to be a decent option for inference due to the high amount of VRAM at its price point.
If it’s completely down to Olive and DirectML then nvidia should be able to use them for similar performance improvements. If not, then AMD is still a defining factor. A quick search didn’t bring up anything definitive on the question though, so I guess we’ll have to wait for someone to try it out ( or someone with faster Google-fu than me)
Well the problem is that Automatic1111 is not fast...
Other diffusers based UIs with PyTorch Triton will net you 40%+ performance.
Facebook AITemplate inference in VoltaML will be at least twice as fast as A1111 on a 3080, with support for LORAs, controlnet and such. This supports AMD Instinct cards too.
What I am getting at is that people dont really care about A1111 performance on a 3080 because, for the most part, its fast enough.
SegmentAnything is the big one missing from other UIs, but IMO most of the other extensions are pretty niche, especially with how hackable diffusers is compared to the A1111/Comfy SAI backend.
From Geohotz's investigation in the matter, it doesn't look like it's a manpower issue, it's a quality/culture issue. AMD's firmware GPU division isn't amazing.
But, it's worth noting there's different levels of "support." With ROCm 5.6, the 7900XT and 7900XTX RDNA3 cards, while not "officially" supported are gfx1100, which have rocBLAS/MIOpen kernels and work w/o jiggerpokery w/ PyTorch nightly and various HIPified inferencers I've tried like ExLlama.
Been watching this quite closely.
As far as I summarise, the 7900XTX is the first (and only) desktop GPU from AMD that _might_ be worth buying. (They own the console gaming space, but thats a different story).
Not Nvidia beating due to the CUDA issue, but a massive leap in the right direction.
Intel is also making _some_ progress with its ARC range.
Its going to be happy days for us users if/when AMD/Intel are competitive, and cut some of that monopoly margin off Nvidias pricing, but a way to go yet.
This^ and also there’s always some driver issue on linux for Nvidia cards. It’s one of those things you set it up and then never want to touch it because if you update it. It’s flipping a coin on if you will spend the next few hours trying to figure out what’s wrong.
I keep hearing complaints about NVIDIA driver issues under Linux, so they must be real.
Nevertheless, I have used for twenty years many different NVIDIA cards under Linux, both GeForce and Quadro models, on many different desktop and laptop computers.
I have never seen any problem with the NVIDIA drivers and libraries, everything has worked fine after installation, without needing any special action beyond choosing to install the NVIDIA software. The same was also true under FreeBSD.
The only exception to this was some years ago, on a laptop with NVIDIA Optimus switchable GPUs, where I have lost a couple of days until succeeding to configure how to select between the Intel GPU and the NVIDIA GPU.
I also have some AMD GPUs, older models which still had great FP64 performance, so they are used for computational applications, but with those I had greater problems under Linux, when used with many monitors.
Therefore whether problems with the Linux NVIDIA drivers are encountered must be dependent on complex combinations of hardware and software, so it is impossible to predict whether they will be encountered or not.
Unexpected problems may always happen with any piece of hardware under an open-source operating system, because the manufacturers typically do not provide technical documentation like they did before 1995 and most of them test their hardware only with Windows. NVIDIA certainly provides much better software support for Linux and FreeBSD than the majority of the hardware vendors.
So my point is that it is incorrect to attempt to discourage the Linux users to use NVIDIA due to supposed driver problems, because many of them, perhaps most of them will never encounter any problem.
Only the Intel GPUs can be said to have better Linux support, and they also have the best GPU technical documentation, much better than that of AMD. While NVIDIA has the worst documentation for their hardware, they have excellent documentation for the huge amount of software that they provide freely for their GPUs under Linux.
What is wrong with NVIDIA is neither their software support for Linux nor the documentation for that software, but their pricing policy that always seems based on a greediness that is excessive even for a profit-oriented company.
You used NVIDIA cards for twenty years and never run into an issue where the mainline driver stopped supporting your GPU and you had to manually download community hosted legacy driver family on every kernel upgrade? Gets old quick...
While I have retired the older cards, up to Kepler, I am still using a Maxwell GPU from 9 years ago and it is still well supported, like also my other Pascal, Volta and Turing GPUs, none of which is new.
There might have been NVIDIA GPUs which have been supported for less than 10 years, but I never happened to have one of those.
I have been using mostly Gentoo, and I never had to do anything besides "emerge nvidia-drivers", with the exception of the case that I have already mentioned with the laptop Optimus configuration, which required installing extra programs besides the official NVIDIA drivers.
In all these twenty years I have used only Linux on all desktops and laptops, both at home and at work, and I have also written various OpenGL and CUDA programs, so I have used the NVIDIA cards in many different circumstances, without encountering problems.
Some years ago, I was hoping that I would be able to get rid of the NVIDIA GPUs and use only AMD GPUs, which had a much better computational performance per dollar.
Unfortunately, AMD has split their GPUs into RDNA GPUs that are good only for gaming and CDNA GPUs that are much too expensive for small companies or individuals, so they no longer make any GPU model that could be an upgrade for my old AMD GPUs, so the only choice remains to buy overpriced NVIDIA GPUs, unless the next Intel generation of GPUs will become more competitive for computational tasks.
At least Intel makes a credible effort to compete with CUDA, with their oneAPI environment.
With the prevalence of Wayland, I don't think it's correct to say that most Linux users will never encounter problems with NVIDIA. Things have improved recently but Intel and AMD are still way ahead when it comes to that.
I have not seen any evidence yet that Wayland is an improvement over X Window System, while for NVIDIA there are a large number of available software packages that are known to provide useful functions.
Very slowly, the AMD GPUs become usable with important applications, like Blender, but the problems that can be encountered when trying to use such applications with AMD GPUs and trying to accomplish something concrete are far more annoying than the fact that Wayland may not work well, because one can always choose to avoid Wayland without losing anything.
The open source drivers for Nvidia have sucked for years, meaning you need to use the proprietary drivers. The proprietary drivers frequently break on kernel updates and have historically been problematic to use in combination with Wayland.
Ouch. This hurts, and I amdit to being an NVIDIA fan, but it is very true.
And may your maker help you if you want to do some HW accelerated ML but accidentally upgrade one of the many NVIDIA components (driver, CUDA, CuDNN) to a version not compatible with that one key Python package.
Note that my Nvidia experience is roughly 5 years old. Some things might have changed during that time.
When I was using Nvidia GPU my experience that 50% after a system update which included kernel update, the Nvidia kmod didn't properly rebuild resulting in graphical interface completely non working next time I booted the system. In such situations I had to switch to terminal and enter a couple of commands to ensure the Nvidia driver is also properly updated and the glue layer between kernel and the driver rebuilt. Most of the time that helped but occasionally it didn't and I had to stay on previous kernel version for a week until the issues got solved.
That was a couple of years ago with fedora. On a different distro with less frequent kernel updates and Nvidia drivers being repackaged by the maintainers of distro instead of third party repo like rpmfusion, or a bigger distro that Nvidia tests ahead of time, the experience might have been slightly more smooth.
Currently with AMD open source driver's that's not an issue since the drivers are part of kernel so everything just works and always has compatible versions. I can install any recent distro and it will just work out of the box.
Last year Nvidia open sourced some parts of their driver, but not all of it (it still depends on large user space binary blobs), it might slightly help with the practical aspects of problem above. But I don't know whether distros embraced it and did it actually help (I dropped Nvidia before that).
I don't disagree that it's at least partially more of the fault of Linux kernel for not having stable API, not so much AMD or Nvidia. But for the end user it doesn't matter whose fault it is. AMD chose to play by the rules of Linux, which makes much better end user experience better.
All of this applies only to graphic driver part, GPU compute is a completely different story.
Another big part as others mentioned is wayland support. For a long time Nvidia drivers dindn't support it at all due to not providing certain interface. From what I understand currently it works on the major desktop environments like Gnome and KDE, but due to them adding support for Nvidia specific interface not the other way around. So it still doesn't work on more niche desktop environments which use the common interface supported by rest of drivers.
~12 years ago, before AMD made their official open source drivers situation was completely different. The choice then for AMD was between non official open source drivers which had 50% performance but worked correctly and the closed source AMD drivers that were 2x faster (when they worked) but otherwise buggy. Compared to that Nvidia closed source drivers worked.
On Kubuntu 22.04 with an RTX 30*0, you have to downgrade NVIDIA drivers to version 515 or the system won't boot. Has been like this since launch. This is especially annoying since if you want to install any other NVIDIA software, the installer will try to update the driver version and break the system again.
A friend of mine has a similar issue on Ubuntu where the system only boots 2 out of 3 times.
I also recall having similar issues almost every time when I installed systems with NVIDIA GPUs in the last decade.
Checking the forums, other people are having similar issues.
> The AMD experience on Linux is vastly better than the Nvidia one.
I just wish we had an equivalent of AMD Software on Linux, so I could mess around with the settings more.
For example, I like to limit the GPU to 50-75% of it's total power for ambient heat/cooling reasons, or UPS/PSU/electricity bill reasons when specific games make it hard to cap framerates.
> You see, by default the card controls its own GPU and memory clock values, which means that when idle the GPU draws around 40 W of power. However, if I want to set a limit for how much W in total it can use, it also makes me set the GPU and memory clock values, which will them be fixed: so at idle the GPU will use about 60 W of power.
Oh also ROCm makes me want to pull my hair out. Still use almost all AMD hardware though, except for low power Celeron in netbook for notes.
Sure, you can see the difference if you're carefully looking for it in comparison videos. But when you're actually playing it's usually not noticeable to any meaningful degree. You're gonna get occasional artefacts with both.
As for ray tracing - to be honest, playing at 4k with an RTX 3070 there's very few games where I can turn it on without the game running unacceptably slow even with DLSS.
Ray traced shadows and AO are nice but hardly a deal breaker.
I think it's more the desire to have the latest and greatest tech that makes Nvidia cards desirable, rather than any material difference that you'll actually notice while gaming.
Some good comparison shots of DLSS 2 and FSR 2 in this video.
I mean, you talk about "comparison videos", when in reality the issue is that significant amount of games support DLSS and pretty much noone supports FSR 2.0.
You can do any comparison you want, but it won't help you when your software won't make use of the library your GPU supports.
(Also, we're at DLSS 3.0 now with frame generation which moves the bar higher.)
Seems like it's still better to buy Nvidia then. Nvidia cards support both DLSS and FSR so you can choose the tech that works best with the particular game.
Meanwhile AMD only supports FSR and if the game looks better with DLSS then there's nothing you can do.
AMD cards also aren't that much cheaper so it doesn't make much sense to buy worse tech that you'll be using for years every day to save a few euros.
It's not as if Nvidia is always a better choice. Things turned around a bit when new crop of games (The Last of Us, Hogwarts Legacy, etc) came out, requiring more than 8GB of VRAM. They do work with 8GB, but with lot more stutterging as textures have to be constantly transfered from main RAM. For these games amount of VRAM made a huge difference. At that point in time, AMD had more mid level cards with 12GB (6700) and 16GB (6800) and for considerably less money than equivalent Nvidia, at least in Europe. Nvidia cheapened and stuck to 8GB even for 3070 which was more expensive than 16GB 6080 around here.. This became a big issue and pushed Nvidia to put more VRAM in next generation.
If you actually try and find proof of the 8GB fiasco, there's mostly one source that put together an artificial scenario where they pushed a bottom end card to 4k on newly launched console ports which then got optimized.
The framing here is a little unfair, its DLSS that does not support AMD hardware, not the other way around. This is clearly a deliberate decision by Nvidia. Meanwhile AMD/FSR supports both AMD and Nvidia hardware.
The framing is fair unless you expect NVIDIA to ditch their ML based solution for a class of inferior implementations which in turn can run just about anywhere.
DLSS isn't "NVIDIA flavored FSR", it's a more advanced pipeline that requires significantly more architectural alignment. FSR 3 was going to be the closest thing to an equivalent and won't run everywhere FSR 2 does.
DLSS directly leverages the tensor cores of particular NVidia cards.
Whether or not this is just a marketing gimmick is up to you and your own conclusions on the matter, but AMD doesn't seem to have an equivalent hardware tech for this yet. It seems to be taking steps in the right direction with the WMMA instruction for accelerating matrix multiplication in a similar fashion, but performance so far head-to-head seems lacking, and I haven't seen anything definitive out of AMD suggesting this for gaming purposes.
Have you tried DLSS 3 with frame generation yet? Ray tracing performance hits are mitigated substantially, 4K included. What is more shocking to me is how low they got the latency on it; I have yet to notice perceptible lag when turned on (contrasting this of course with TVs that attempt to increase frames to achieve the 'soap opera effect', where lag becomes substantial). I just hope more games support it one day.
FSR may one day get much better, but in my personal preferences and tastes, I greatly prefer DLSS for upscaling. It tends to be sharper, better at fine details at long distances, and tends to do better with certain (literal) edge cases.
I haven't, like most people I guess it'll be at least half a decade until I upgrade to a 40 series card.
> What is more shocking to me is how low they got the latency on it
Most games are perfectly playable at a stable 30fps (33.3ms latency at least). The frame generation is hopefully giving you 90-120fps. So you're getting latency sub 16.6ms, equivalent to stable 45-60fps. That will usually only be noticeable in drawing apps or twitchy FPS games.
Maybe they're doing some other tricks like decoupling input latency from frame latency, I haven't looked into it. It's common to do this with physics, in the opposite direction (e.g. run physics at a fixed 33.3ms interval while generating frames as fast as possible, aiming for 60fps).
“Nvidia's SDK, CUDA, is the industry standard and they have built a huge range of tools on top of that for training large models, inference-serving, optimization, transfer learning, infrastructure management and operations automation, and a lot of vertical-specific applications.”
So its not the hw, but an API issue? Is it not possible to port the tools to the AMD API? Or does the AMD API not allow to properly utilize the compute power of the hw?
AMDs APIs have historically been way behind CUDA, both in features and ease of use, making it much harder to reach feature and performance parity with CUDA.
The second big problem is that AMD has a history of abandoning APIs. While Nvidia has been focusing on CUDA from the start, AMD is on its third or fourth API. A lot of developers got burned early on by putting in a lot of effort into supporting AMD cards, only to have their work become useless as AMD dropped old API.
So if you wanted to support AMD today you have to choose between using their latest API which is currently only supported on a handful of their latest top end cards, or using their previous API, which is much worse and may or may not be supported in 2 years time.
> their latest API [...] is currently only supported on a handful of their latest top end cards
While there's very few cards that are officially supported, I've enabled most RDNA 1 and RDNA 2 GPUs in the OS-provided packages on Debian Unstable and Ubuntu 23.10. I haven't tested every architecture, but my experience has been that they generally work quite well. The version of HIP packaged for Debian is still a few versions behind, so we haven't been able to enable RDNA 3 yet, but it shouldn't be too long before that is resolved.
As you have described, it has been a very long road to get to this point (with a lot of mistakes along the way), but I think things are getting better. That said, I work on ROCm, so my evaluation may be unintentionally biased.
Hopefully. Although I haven't programmed with it yet, the latest ROCm looks nice (at least from reading the docs). Now all you have to do is actually get to a point where it's actually supported on hardware people actually own and convince people that it's worth investing their time in, and then AMD can start making progress.
The Debian packages support a much wider range of hardware than the AMD packages do. Just about any discrete AMD GPU hardware released in the past five years should already work or will work soon.
CUDA is proprietary to Nvidia, and it’s unlikely they would ever allow implementing it on other vendors’ hardware. DirectML mentioned here is proprietary to Microsoft but at least hardware-neutral. It is also much less popular. OpenCL is open but receives hardly any usage at all, unfortunately.
(To be fair, Nvidia were at the forefront of this GPGPU thing since before there were dedicated APIs for it, so they certainly deserve some credit, but at this point they seem in desperate need for someone to finally knock them down.)
Nvidia has provided the same API incrementally extending it since 2007. And supported it on most of their GPUS including consumer cards. Oldest cards supported by the latest version of their SDK were made in ~2013-2016. See table in https://en.wikipedia.org/wiki/CUDA#GPUs_supported
AMD has been a lot less consistent with their support changing their mind almost every 3 years. ROCm was initially released in 2016 (that's 9 years of headstart for NVIDIA). If you look at https://docs.amd.com/en/latest/release/gpu_os_support.html on Linux only the enterprise server cards are supported. Situation slightly different on windows https://docs.amd.com/en/latest/release/windows_support.html where they support last 2 generations of consumer cards (partially). I remember there were also a few years with even more bizarre situation where they supported GPUs that are couple of generations old but not their latest cards.
Not hard to see how developers are hesitant to spend time adding support for AMD if large portion of AMD users won't be able to use it and there is a high chance that after few years AMD will change their mind again coming out with new API or dropping support for cards that are only couple of years old.
CUDA has over a decade of tooling and talent built up while AMD/Apple fumbled with the far more awkward to develop with OpenCL that they eventually killed.
The software support for AMD gpu is probably lags 10 years or so after Nvidia. They just release there sdk for windows this year (more specifically 2023-05-24, Rocm 5.5.1). While CUDA exists for both platform like forever. So there won't be a common user group besides some lab (which have the man power to invest in linux software and setup new tool chains) that use AMD card for AI related task.
But since Rocm for windows finally released. The situation may finally change. We can probably just wait and see what will go on.
My impression was rather that CUDA is successful because it will work on any NVidia GPU that's not yet old enough to vote, which means a number of things:
- people had lots of time to get used to it
- hobbyists and smaller research outfits/labs could make use of it on the cheap and less powerful hardware (where even a cheap GPGPU beats computing in software), driving:
- mind share (compare: Photoshop/Microsoft's tolerance to piracy)
- sales of hardware when the users decided to upgrade, buying more and/or and/or more modern and/or better cards
- the longer it has been available, the more stable of a programming target it appears to be, driving a virtuous cycle
In contrast, the range of AMD hardware supporting ROCm was narrow and unstable (AFAIK one card was only supported for a couple of years). This would of course make ROCm unattractive for building with, especially for the hobbyists and smaller outfits that would otherwise drive mindshare but couldn't afford datacenter grade hardware that was supported.
Recent rocm on windows also supports more cards than it used to be. Consumer rdna2/3 cards are mostly on the support list. Although these only available on windows for some reason. And the only consumer card supported on linux is still 'AMD Radeon™ VII'.
No idea if AMD will drop whole list like it previously done this time though.
Its a stable market. Its not growing like the AI market is and Nvidia makes much higher margins by selling GPUs to the enterprise market. Gaming is bound to become a tiny piece of the pie.
ML hardware sales are being driven by a 'Renaissance' of rapid model development and commercial offerings, primarily by the FAANG, but several others too. This is neat, but beyond the initial waves, OpenGPT isn't exactly the digital messiah many suggested it was, and running these models is expensive (both in HW and power for it). Time will tell, but I am skeptical this will be more than a craze for most applications (except digital art, which is awesome and has already been proven effective; just not maybe as marketable to the masses).
AI market just caught up with gaming this year. But aa it's growing faster than gaming, in a few years GPUs have to improve more in AI performance (matmul ops and interconnect) than gaming.
I'm actually curious if libraries like pytorch are even trying to move away from CUDA and if moving away from it is worth it. I get why newer ML toolchains would do that but do mainstream established MI frameeorks plan on sticking with nvidia exclusivity for now?
Start with Pytorch; if you removed CUDA support, all that's left is hacks honestly. Stuff like Olive and DirectML is equally as proprietary as CUDA while also less-developed, and Metal Performance Shaders are a hacked wrapper for GPU compute. See for yourself: https://pytorch.org/docs/stable/backends.html
There's also ONNX, a Pytorch alternative made by Microsoft that focuses on optimizing for as many targets as possible. It is very difficult to use though, and still under active development.
Generally, the issue is that our FAANG companies hate each other too much to do anything about Nvidia. AMD and Apple did have a working relationship (and even a GPGPU acceleration library), but now they are bitter enemies.
I've been running pytorch and rocm (5.6 has support for gfx1100 if you compile it yourself) for at least 3 months at 18 it/s on a 7900 XTX. This has been possible for quite a while.
Could someone fill me in on what's actually new here, other than the specific technology used?
This is hard to quantify, there are no benchmarks for how much time people waste to get things to (not) run. In my experience, hardware stability is fine, but software stack has always been problematic. Just the way they treat ROCm makes me stay away. If you have a multi million dollar contract for some datacenter rollout, maybe the opportunity cost amortizes. Maybe.
I am gaming daily on 7900XTX, can you tell me what is not stable? I mean yes, it was not perfect for the first two months I had it, but now I dont have any issues (driver 23.7.1). I have even tried gaming on linux.
123 comments
[ 2.3 ms ] story [ 212 ms ] threadGranted maybe not all extensions are compatible with DirectML in Automatic1111.
- AMD's support is flaky. The AMD 7900 XTX is only officially supported on Windows PCs (for ROCm). On Linux PCs in my experience there is a high risk of graphics card hard lockups.
- Historically AMD has dropped support for consumer cards really quickly (I don't think they support any consumer cards from more than about 3 years ago). At some point that'll stabilise, but it is hard to tell if this is the generation that will start working long term or the next.
- AMD generally gets second class support in return from the machine learning communities. There still seems to be some chance that you'll be locked out of the latest and greatest if you go AMD.
At some point this will all probably come good and it'll be like CPUs where there is really no difference between the vendors. There is too much interest in machine learning right now for anything else to happen ... but I've been buying AMD for more than a decade now and I've been thinking that for years already. If someone is on the edge they'd be better off not getting seduced by a minor performance improvement and sticking to Nvidia cards. It is uncertain when the long term will reach us. This year? 5 years? A decade? All options where AMDs software drivers are in play.
Although AMDs graphics drivers do seem great on Linux nowadays, I like things that just work. It is a pity they got so out of position vs CUDA.
What do you mean, does this become a crash / kernel panic? Is this a common occurrence and is it due to bad kernel drivers?
https://github.com/RadeonOpenCompute/ROCm/issues/2198
https://geohot.github.io/blog/jekyll/update/2023/06/07/a-div...
they don't occur in amdgpu-pro proprietary userland, but AMD apparently runs that on an internal release cycle and doesn't make the current repo or build available to the public. And AMD just historically hasn't cared enough about the open userland to support that even in the officially-supported configurations.
I'll also add to GP by saying that the windows support is all of about 2 weeks old at this point and it's also the first time that consumer cards (as opposed to the CDNA compute cards and the workstation-branded Radeon Pro cards) have had official support. Not that that is a guarantee it'll actually run with AMD though, especially on the open-source userland.
In which case, I would recommend the AITemplate extension[0] for ComfyUI, which runing at a (hopefully) standardized 512x512 res, default euler A sampler, nets me about 23.8 it/s on my 350w 3090.
[0] https://github.com/FizzleDorf/AIT
With olive in general, you need to rebuild all the models for it; and a whole lot of the extensions for A1111 are support for workflow components with new models (often, whole new classes of models). Unless Olive becomes a major platform for people developing models, this is always going to be lagging.
This is a very strange comparison.
I mean, they announced it with a more than 2x speedup for SD in May:
https://blogs.nvidia.com/blog/2023/05/23/microsoft-build-nvi...
So the headline should be Microsoft Olive vs. PyTorch and not AMD vs. Nvidia.
I’m not sure which customer willing to spend $1000-1200 to do ML workflows isn’t willing to spend $1600 to get another 20%+ of performance and have the fastest card available.
I’m not saying people have unlimited budgets but it just seems like the choice most people in that price range would make.
Tesla P40 is being heavily discussed in these circles because although it's janky (extremely bad TFLOPS for 16-bit operations), it's still thought to be a decent option for inference due to the high amount of VRAM at its price point.
Other diffusers based UIs with PyTorch Triton will net you 40%+ performance.
Facebook AITemplate inference in VoltaML will be at least twice as fast as A1111 on a 3080, with support for LORAs, controlnet and such. This supports AMD Instinct cards too.
What I am getting at is that people dont really care about A1111 performance on a 3080 because, for the most part, its fast enough.
You also have to keep in mind some latest gen AMD GPUs don’t even officially support ROCm on Linux. That’s absurd.
AMD has a choice to invest more staff into ML support, they’re choosing not to.
But, it's worth noting there's different levels of "support." With ROCm 5.6, the 7900XT and 7900XTX RDNA3 cards, while not "officially" supported are gfx1100, which have rocBLAS/MIOpen kernels and work w/o jiggerpokery w/ PyTorch nightly and various HIPified inferencers I've tried like ExLlama.
Not Nvidia beating due to the CUDA issue, but a massive leap in the right direction.
Intel is also making _some_ progress with its ARC range.
Its going to be happy days for us users if/when AMD/Intel are competitive, and cut some of that monopoly margin off Nvidias pricing, but a way to go yet.
Unless you need RT for gaming then most of AMDs cards are better value.
Nevertheless, I have used for twenty years many different NVIDIA cards under Linux, both GeForce and Quadro models, on many different desktop and laptop computers.
I have never seen any problem with the NVIDIA drivers and libraries, everything has worked fine after installation, without needing any special action beyond choosing to install the NVIDIA software. The same was also true under FreeBSD.
The only exception to this was some years ago, on a laptop with NVIDIA Optimus switchable GPUs, where I have lost a couple of days until succeeding to configure how to select between the Intel GPU and the NVIDIA GPU.
I also have some AMD GPUs, older models which still had great FP64 performance, so they are used for computational applications, but with those I had greater problems under Linux, when used with many monitors.
Therefore whether problems with the Linux NVIDIA drivers are encountered must be dependent on complex combinations of hardware and software, so it is impossible to predict whether they will be encountered or not.
Unexpected problems may always happen with any piece of hardware under an open-source operating system, because the manufacturers typically do not provide technical documentation like they did before 1995 and most of them test their hardware only with Windows. NVIDIA certainly provides much better software support for Linux and FreeBSD than the majority of the hardware vendors.
So my point is that it is incorrect to attempt to discourage the Linux users to use NVIDIA due to supposed driver problems, because many of them, perhaps most of them will never encounter any problem.
Only the Intel GPUs can be said to have better Linux support, and they also have the best GPU technical documentation, much better than that of AMD. While NVIDIA has the worst documentation for their hardware, they have excellent documentation for the huge amount of software that they provide freely for their GPUs under Linux.
What is wrong with NVIDIA is neither their software support for Linux nor the documentation for that software, but their pricing policy that always seems based on a greediness that is excessive even for a profit-oriented company.
There might have been NVIDIA GPUs which have been supported for less than 10 years, but I never happened to have one of those.
I have been using mostly Gentoo, and I never had to do anything besides "emerge nvidia-drivers", with the exception of the case that I have already mentioned with the laptop Optimus configuration, which required installing extra programs besides the official NVIDIA drivers.
In all these twenty years I have used only Linux on all desktops and laptops, both at home and at work, and I have also written various OpenGL and CUDA programs, so I have used the NVIDIA cards in many different circumstances, without encountering problems.
Some years ago, I was hoping that I would be able to get rid of the NVIDIA GPUs and use only AMD GPUs, which had a much better computational performance per dollar.
Unfortunately, AMD has split their GPUs into RDNA GPUs that are good only for gaming and CDNA GPUs that are much too expensive for small companies or individuals, so they no longer make any GPU model that could be an upgrade for my old AMD GPUs, so the only choice remains to buy overpriced NVIDIA GPUs, unless the next Intel generation of GPUs will become more competitive for computational tasks.
At least Intel makes a credible effort to compete with CUDA, with their oneAPI environment.
Very slowly, the AMD GPUs become usable with important applications, like Blender, but the problems that can be encountered when trying to use such applications with AMD GPUs and trying to accomplish something concrete are far more annoying than the fact that Wayland may not work well, because one can always choose to avoid Wayland without losing anything.
And may your maker help you if you want to do some HW accelerated ML but accidentally upgrade one of the many NVIDIA components (driver, CUDA, CuDNN) to a version not compatible with that one key Python package.
When I was using Nvidia GPU my experience that 50% after a system update which included kernel update, the Nvidia kmod didn't properly rebuild resulting in graphical interface completely non working next time I booted the system. In such situations I had to switch to terminal and enter a couple of commands to ensure the Nvidia driver is also properly updated and the glue layer between kernel and the driver rebuilt. Most of the time that helped but occasionally it didn't and I had to stay on previous kernel version for a week until the issues got solved.
That was a couple of years ago with fedora. On a different distro with less frequent kernel updates and Nvidia drivers being repackaged by the maintainers of distro instead of third party repo like rpmfusion, or a bigger distro that Nvidia tests ahead of time, the experience might have been slightly more smooth.
Currently with AMD open source driver's that's not an issue since the drivers are part of kernel so everything just works and always has compatible versions. I can install any recent distro and it will just work out of the box.
Last year Nvidia open sourced some parts of their driver, but not all of it (it still depends on large user space binary blobs), it might slightly help with the practical aspects of problem above. But I don't know whether distros embraced it and did it actually help (I dropped Nvidia before that).
I don't disagree that it's at least partially more of the fault of Linux kernel for not having stable API, not so much AMD or Nvidia. But for the end user it doesn't matter whose fault it is. AMD chose to play by the rules of Linux, which makes much better end user experience better.
All of this applies only to graphic driver part, GPU compute is a completely different story.
Another big part as others mentioned is wayland support. For a long time Nvidia drivers dindn't support it at all due to not providing certain interface. From what I understand currently it works on the major desktop environments like Gnome and KDE, but due to them adding support for Nvidia specific interface not the other way around. So it still doesn't work on more niche desktop environments which use the common interface supported by rest of drivers.
~12 years ago, before AMD made their official open source drivers situation was completely different. The choice then for AMD was between non official open source drivers which had 50% performance but worked correctly and the closed source AMD drivers that were 2x faster (when they worked) but otherwise buggy. Compared to that Nvidia closed source drivers worked.
A friend of mine has a similar issue on Ubuntu where the system only boots 2 out of 3 times.
I also recall having similar issues almost every time when I installed systems with NVIDIA GPUs in the last decade.
Checking the forums, other people are having similar issues.
https://forums.developer.nvidia.com/t/no-gui-or-display-afte...
https://forums.developer.nvidia.com/t/no-display-output-when...
I just wish we had an equivalent of AMD Software on Linux, so I could mess around with the settings more.
For example, I like to limit the GPU to 50-75% of it's total power for ambient heat/cooling reasons, or UPS/PSU/electricity bill reasons when specific games make it hard to cap framerates.
With AMD Software on Windows, it's no big deal. On Linux, the best I found was CoreCtrl: https://gitlab.com/corectrl/corectrl
Sadly, it doesn't seem to work all that well for my use case, which I mentioned in my blog post when using Linux instead of Windows as my daily driver at home too: https://blog.kronis.dev/articles/a-week-of-linux-instead-of-...
> You see, by default the card controls its own GPU and memory clock values, which means that when idle the GPU draws around 40 W of power. However, if I want to set a limit for how much W in total it can use, it also makes me set the GPU and memory clock values, which will them be fixed: so at idle the GPU will use about 60 W of power.
Oh also ROCm makes me want to pull my hair out. Still use almost all AMD hardware though, except for low power Celeron in netbook for notes.
Nobody "needs" RT for gaming. It's still in gimmicky phase and not worth neither performance hit nor the way it looks on screen.
Edit: of course I am talking about gaming here since you mentioned consoles
For me the problem is technology and not raw performance
DLSS and RT are massive for someone with a 4k screen but now 4k gaming hardware outside league of legends lol
Sure, you can see the difference if you're carefully looking for it in comparison videos. But when you're actually playing it's usually not noticeable to any meaningful degree. You're gonna get occasional artefacts with both.
As for ray tracing - to be honest, playing at 4k with an RTX 3070 there's very few games where I can turn it on without the game running unacceptably slow even with DLSS.
Ray traced shadows and AO are nice but hardly a deal breaker.
I think it's more the desire to have the latest and greatest tech that makes Nvidia cards desirable, rather than any material difference that you'll actually notice while gaming.
Some good comparison shots of DLSS 2 and FSR 2 in this video.
https://youtu.be/1WM_w7TBbj0
You can do any comparison you want, but it won't help you when your software won't make use of the library your GPU supports.
(Also, we're at DLSS 3.0 now with frame generation which moves the bar higher.)
FSR2 needs game specific integration (IIRC because it needs access to motion vectors) and cannot be applied globally.
- the overwhelming conclusion is DLSS is the superior upscaling technology with near universally better results
- it was a brutal result for AMD
- DLSS gives NVIDIA a clear selling point
I'm not sure if it backs your suggestion...
Meanwhile AMD only supports FSR and if the game looks better with DLSS then there's nothing you can do.
AMD cards also aren't that much cheaper so it doesn't make much sense to buy worse tech that you'll be using for years every day to save a few euros.
NVIDIA said as much, released a 16GB version, and lo-and-behold it's absolutely pointless, even performing worse than the 8GB at times: https://www.tomshardware.com/reviews/nvidia-geforce-rtx-4060...
Most graphics cards are still under 8GB of VRAM, and raw numbers of memory storage mean nothing for performance in isolation.
DLSS isn't "NVIDIA flavored FSR", it's a more advanced pipeline that requires significantly more architectural alignment. FSR 3 was going to be the closest thing to an equivalent and won't run everywhere FSR 2 does.
Whether or not this is just a marketing gimmick is up to you and your own conclusions on the matter, but AMD doesn't seem to have an equivalent hardware tech for this yet. It seems to be taking steps in the right direction with the WMMA instruction for accelerating matrix multiplication in a similar fashion, but performance so far head-to-head seems lacking, and I haven't seen anything definitive out of AMD suggesting this for gaming purposes.
FSR may one day get much better, but in my personal preferences and tastes, I greatly prefer DLSS for upscaling. It tends to be sharper, better at fine details at long distances, and tends to do better with certain (literal) edge cases.
> What is more shocking to me is how low they got the latency on it
Most games are perfectly playable at a stable 30fps (33.3ms latency at least). The frame generation is hopefully giving you 90-120fps. So you're getting latency sub 16.6ms, equivalent to stable 45-60fps. That will usually only be noticeable in drawing apps or twitchy FPS games.
Maybe they're doing some other tricks like decoupling input latency from frame latency, I haven't looked into it. It's common to do this with physics, in the opposite direction (e.g. run physics at a fixed 33.3ms interval while generating frames as fast as possible, aiming for 60fps).
https://www.google.com/search?q=why%20is%20nvidia%20better%2...
The second big problem is that AMD has a history of abandoning APIs. While Nvidia has been focusing on CUDA from the start, AMD is on its third or fourth API. A lot of developers got burned early on by putting in a lot of effort into supporting AMD cards, only to have their work become useless as AMD dropped old API.
So if you wanted to support AMD today you have to choose between using their latest API which is currently only supported on a handful of their latest top end cards, or using their previous API, which is much worse and may or may not be supported in 2 years time.
While there's very few cards that are officially supported, I've enabled most RDNA 1 and RDNA 2 GPUs in the OS-provided packages on Debian Unstable and Ubuntu 23.10. I haven't tested every architecture, but my experience has been that they generally work quite well. The version of HIP packaged for Debian is still a few versions behind, so we haven't been able to enable RDNA 3 yet, but it shouldn't be too long before that is resolved.
As you have described, it has been a very long road to get to this point (with a lot of mistakes along the way), but I think things are getting better. That said, I work on ROCm, so my evaluation may be unintentionally biased.
Hopefully. Although I haven't programmed with it yet, the latest ROCm looks nice (at least from reading the docs). Now all you have to do is actually get to a point where it's actually supported on hardware people actually own and convince people that it's worth investing their time in, and then AMD can start making progress.
(To be fair, Nvidia were at the forefront of this GPGPU thing since before there were dedicated APIs for it, so they certainly deserve some credit, but at this point they seem in desperate need for someone to finally knock them down.)
AMD has been a lot less consistent with their support changing their mind almost every 3 years. ROCm was initially released in 2016 (that's 9 years of headstart for NVIDIA). If you look at https://docs.amd.com/en/latest/release/gpu_os_support.html on Linux only the enterprise server cards are supported. Situation slightly different on windows https://docs.amd.com/en/latest/release/windows_support.html where they support last 2 generations of consumer cards (partially). I remember there were also a few years with even more bizarre situation where they supported GPUs that are couple of generations old but not their latest cards.
Not hard to see how developers are hesitant to spend time adding support for AMD if large portion of AMD users won't be able to use it and there is a high chance that after few years AMD will change their mind again coming out with new API or dropping support for cards that are only couple of years old.
This disparity is due to that fumble.
But since Rocm for windows finally released. The situation may finally change. We can probably just wait and see what will go on.
- people had lots of time to get used to it
- hobbyists and smaller research outfits/labs could make use of it on the cheap and less powerful hardware (where even a cheap GPGPU beats computing in software), driving:
- the longer it has been available, the more stable of a programming target it appears to be, driving a virtuous cycleIn contrast, the range of AMD hardware supporting ROCm was narrow and unstable (AFAIK one card was only supported for a couple of years). This would of course make ROCm unattractive for building with, especially for the hobbyists and smaller outfits that would otherwise drive mindshare but couldn't afford datacenter grade hardware that was supported.
No idea if AMD will drop whole list like it previously done this time though.
ML hardware sales are being driven by a 'Renaissance' of rapid model development and commercial offerings, primarily by the FAANG, but several others too. This is neat, but beyond the initial waves, OpenGPT isn't exactly the digital messiah many suggested it was, and running these models is expensive (both in HW and power for it). Time will tell, but I am skeptical this will be more than a craze for most applications (except digital art, which is awesome and has already been proven effective; just not maybe as marketable to the masses).
doing?
Start with Pytorch; if you removed CUDA support, all that's left is hacks honestly. Stuff like Olive and DirectML is equally as proprietary as CUDA while also less-developed, and Metal Performance Shaders are a hacked wrapper for GPU compute. See for yourself: https://pytorch.org/docs/stable/backends.html
There's also ONNX, a Pytorch alternative made by Microsoft that focuses on optimizing for as many targets as possible. It is very difficult to use though, and still under active development.
Generally, the issue is that our FAANG companies hate each other too much to do anything about Nvidia. AMD and Apple did have a working relationship (and even a GPGPU acceleration library), but now they are bitter enemies.
Could someone fill me in on what's actually new here, other than the specific technology used?
These days AMD has even better drivers with lower overhead than Nvidia https://youtu.be/H9guEsBly0I