That's practically the TDP for an entire x1 chassis though. Check out the TDP for a H100 DGX Pod. The performance per watt of GH200 *should* be a massive improvement over x86 with Hopper, especially for clusters
While LLMs is what cool kids all talk about nowadays, there's way, way, way more to the machine learning than them. I'd say that minority of GPUs are used for LLMs in the ML space. There's likely way more used them where they directly bring billions of dollars to the companies (like all the rankers across big tech, for content, ads, search, etc).
It's not really clear what you mean, maybe there's gaps in my ML knowledge, but generally the answer depends on "what throughput do you need for your use case?"
Generally I agree that if you only need to deliver mail once daily on a mile long route, a car that takes 24 hours to travel a mile is fine.
Sorry, it was a rude comment to make based on some strange grammar I noticed. I compared with some of your prior comments and noticed you didn’t have any such issues before. I wasn’t being facetious or judgemental - genuinely curious. There’s not a non rude way to ask that, should have kept it to myself.
Although it is not as hyped, HPC is much wider than just machine learning. H100 are incredibly useful in numerical simulation and scientific computing.
Already answered below, but the bulk of the market for H100’s appears to be AI. HPC was historically the largest market for this class of accelerator, but it appears to have shifted to hyper scale cloud computing centers.
HPC’s are used to distribute the computation of differential equations using numerical methods and AI appears to be making inroads here as well. I would be interested in others’ opinion on whether it’s a settled question on whether or not AI will be able to produce equivalent solutions.
It's an iterative thing! Neural Network based surrogate models are becoming popular, but you need the compute to solve the PDEs to train the AI and compute the loss in the first place ;)
Nvidia, Please make CUDA open source and compatible with all other GPUs. What you are doing currently is anti-competance and I wish Government passed Anti-Trust Regulations over this.
Why should they do that? They created the technology and they have every right to make money from it. It is not NVIDIA's fault that their competitors offerings are lame, and it's not their duty to give them a helping hand.
EDIT: Also, NVIDIA is nowhere near the market share that would warrant ant-trust action against them.
24 comments
[ 3.0 ms ] story [ 185 ms ] threadI can’t really see another market where it’d make sense.
0. https://www.theverge.com/23894647/amd-ceo-lisa-su-ai-chips-n...
Also useful as a space heater.
While LLMs is what cool kids all talk about nowadays, there's way, way, way more to the machine learning than them. I'd say that minority of GPUs are used for LLMs in the ML space. There's likely way more used them where they directly bring billions of dollars to the companies (like all the rankers across big tech, for content, ads, search, etc).
Are recommendation/ranking models large enough to take advantage of this? I don't think these cards are generally competitive in throughput/$.
Generally I agree that if you only need to deliver mail once daily on a mile long route, a car that takes 24 hours to travel a mile is fine.
To my knowledge, most inferencing (at least for simpler models) happens on cheaper, slower GPUs that have better throughput/dollar.
HPC’s are used to distribute the computation of differential equations using numerical methods and AI appears to be making inroads here as well. I would be interested in others’ opinion on whether it’s a settled question on whether or not AI will be able to produce equivalent solutions.
EDIT: Also, NVIDIA is nowhere near the market share that would warrant ant-trust action against them.