The Quarterly Revenue Trend[1] shows that the "Gaming" category is once again responsible for most of the revenue loss: $1,574M this quarter vs $2,042 in Q2. Of course, the "Gaming" category is known to includes crypto mining.
Searching on eBay for "RTX 3080" and limiting it to sold listings, I see a lot having sold in the last few days for ~$500. Based on how GPU MSRPs have been going, seems like a decent price to me. I was never sure what to expect post-GPU-crypto, were you expecting cheaper?
Not sure how much crypto mining going away affects it though, demand is still there, you just can't buy their cards right now. I've been trying to buy a few rtx 4090s since launch day and it's nowhere to be found.
On the off chance you get to put it in your cart, it either disappears by the time you check out or the order gets voided afterwards. It rarely stays in stock longer than a few mins.
An AIB supplier or an AIB partner is a company that buys the AMD (or Nvidia) Graphics Processor Unit to put on a board and then bring a complete and usable Graphics Card or AIB to market.
Yes, apologies! I'm so deep in the AMD/Intel/Nvidia rumor mill I forget how normal humans talk, sometimes.
Tech news people tend to use "AIBs" as shorthand for "second-party (add-in) board partners" -- that is, companies that sell graphics cards using AMD or Nvidia GPUs but with their own board and cooler designs. So in the case of Nvidia this would be EVGA, Gigabyte, MSI, Zotac, ASUS, etc. But crucially NOT including Nvidia themselves, even though Nvidia sell their own reference design they call the "Founder's Edition".
For machine learning, self funding my startup and can't afford the expensive server grade gpus. Other countries are usually above msrp + shipping + import tax, would be cheaper to buy from a scalper.
Depending on the country you could try to find a hardware swap subreddit. Mining RTX 3090 (not the one that you mentioned but still) often go for <1000 CAD so you could get 2.5 RTX 3090 for each RTX 4090 you wanted.
There are plenty of 3090s on craigslist as well, fairly common to find below 800$, but 4090s are more power efficient and it's easier to manage a single rig with 7 gpus than trying to network 2 rigs with 14 gpus total. It's also faster for inference without any model change.
What is surprising is the Data Centre figure is by far the largest component of their revenue, more than twice that of "gaming" and close to double what it was in Q4 FY21.
That's phenomenal growth for a product category that was a very short time ago almost exclusively desktop-only. I guess ML workloads are huge and here to stay.
A 8xH100 server sells for like 300k+$ (the A100s go for half, still very expensive), I'd imagine the margins are pretty high on that. The demand for ML models (therefore servers) will likely grow now that startups seem to switch from crypto hype to ai hype and big tech derives huge value from ml workloads.
Is this possible: the "crypto hype" and the "ai hype" and this nvidia quarterly earnings all intersecting at the need for high-end chips running on a bunch of servers.
In a supposed down economy c. early next year, offering processor power and hard disk space from home machines as a source of passive income could be the next OnlyFans.
I don't see how crowdsourcing compute would be possible without giving access to all the data being crunched, which would be a complete deal-breaker for many (most?) businesses, I would imagine. So unless someone can figure that part out, it's probably a non-starter.
With disk space, you have the issue of liability—what happens when someone decides to store child pornography on my hard drive? That's a more solvable problem, but I'm certainly not envious of anyone who wants to tackle it.
Yep. If Nvidia can sell their datacenter cards (not even full servers) the revenue is basically 10x for nearly equivalent silicon (>$12k direct b2b price vs $1600 through retail channels per card). All they have to do is switch their semiconductor orders to the server chips and for the 8x servers build some interconnect boards.
The reason you can’t find 4090s is because Nvidia no longer cares about retail consumers.
> "I guess ML workloads are huge and here to stay."
I think so but it deserves a bit of a caveat: a lot of ML workloads right now are distinctively unprofitable, and like all of the unprofitable-venture-funded businesses of the past decade, many of which have been culled in the recent downturn, there will have to be a reckoning at some point.
There are definitely lots of workloads that produce positive ROI for companies, but many more that are heavily subsidized (ex. products like Alexa and Google Assistant) and consume a vast amount of ML resources.
A lot of ML-centric products suffer from a significant departure from the traditional Silicon Valley notion of each additional user being zero marginal cost, and products having negligible operating costs vs. high fixed costs.
The interesting thing about ML workloads is that they require heavy upfront GPU power, but the compute requirements to run the model are much smaller after training.
But all existing ML models are still pretty clearly inadequate for their purpose, and require vastly more training. Meaning you can't just stop investing in hardware or your competitors will have a much better model than you in a couple of years.
As a researcher, I think we'll go into a "winter" soon, but nowhere near the previous one. The hype will die down but ML has shown to be extremely useful. Anything with graphic design, gaming, video, imaging, streaming, etc uses ML these days. It's been used in science for awhile (under statistical learning, e.g. regression) and we can't do without it. Though a lot of these things aren't following the main hype machine. But if you're in the weeds you'll see a lot of these works.
But I do think the hype will die. People even now are starting to notice that a lot of the diffusion work looks amazing at first glance but has major errors. There's also the issue of alignment. These are going to likely take a lot of mathematical experience to push forward[0] and will benefit less from a pure numbers game.
Of course, the other part (in Nvidia's favor) is that a lot of scientific work is GPU based now. Not just ML. Parallelism is king. You can do FEM, particle simulations, etc with GPUs now because the clock speed got high enough. So you can do "heavy" calculations with extreme parallelism. Previously you could only do this with lighter loads. (there's also major algorithmic improvements, so I don't want to undermine the fantastic work of those researchers).
[0] I want to admit my bias here. I'm a "math person" (not a mathematician, they will school me) and there's not a lot of ML researchers going down this path. To give an example, there are plenty of diffusion researchers I've talked to that don't know how to calculate the likelihood of their samples while they use an ELBO cost function. But this comes into the alignment issue because we need to talk about the smoothness of latent manifolds, interpolation, inversion (not just image inversion, but image/text inversion), and a lot more. I just don't think we can continue on this path of bigger networks, bigger data, search hyper-parameters, get benchmark result. (I fundamentally believe this is anti-scientific fwiw)
> Of course, the other part (in Nvidia's favor) is that a lot of scientific work is GPU based now. Not just ML.
It’s not just the hardware like you’re talking about. The tooling has grown up insanely well too. You used to have to write CUDA or similar if you wanted a simulation on a GPU. Now you can write it in a high level language in a way that looks pretty close to the math.
Very true! I should have appended this under the algorithmic improvements. This is also a reason ML has exploded. Tensorflow and Pytorch enabled easy GPU usage and so we can spend a lot less time programming and debugging. (even writing CUDA subroutines is easier!) I mean pytorch is basically numpy but with GPU access so it is fantastic for any optimization, even non ml. I haven't played around with JAX but I hear it is better for more statistical stuff because of this.
My only caution on that is that I think their moat on ML is much shallower than people think. nVidia has traditionally maintained their position as the top dog in GPGPU by being first to provide good hardware and software, having markets lock in on that software, and then able to later sell more hardware at very good margins. Even when AMD (and others) later provide hardware that provides much better perf/cost for the same workload, at that point everyone already has a small mountain of tightly optimized cuda, and the cost/benefit of switching hardware providers just isn't there.
I really don't think this is going to work in ML. At their core, the ML algorithms are generally simple enough to fit on a blackboard, and are relatively much more easy to efficiently parallelize and optimize on diverse hardware. nVidia has again managed to capture most of a nascent market by being the first with adequate hardware and software to support it. But I think in a couple of years there are going to be a lot more competitors in this space, and unlike with previous GPGPU work, it will not be that painful for the end-user to shop around.
This won't mean that AMD (or someone else! this space is much easier to enter than GPUs in general) will capture the value from nVidia. Instead it will mean that the entire space will be commoditized and the margins will absolutely collapse.
The same should have been true about Bitcoin mining, but 32nm ASIC miners would get smoked by Nvidia cards in price, performance, and power efficiency. Nvidia's real power here is that they're the only company competing with Apple for the latest TSMC silicon. Their ability to drop 9 billion dollars on a foundry investment secured them the 4nm node, which is their bid for future-proofing their products. So far, it's worked.
I think the inertia of switching is lower for ML because libraries like pytorch support more backends besides just CUDA, such as ROCm (AMD) and MPS (Apple Metal). I used the pytorch ROCm backend recently and there's certainly some more work that needs to be done in this space to get it to the same level as the CUDA backend (performance and compatibility wise), but at least it makes other GPU vendors an option.
This. Most ML people we work with, including new hires, are CPU-first and don't think in bandwidth/throughput, so we have to untrain basically every DS hire to this. Nvidia SW+HW is so far ahead here as they got it early. Sure, Intel will pay startups to port individual algs to make some steps do well, but the ecosystem gap is just getting bigger. AMD/Intel/etc are still playing catch-up to status quo from years ago, while we work with Nvidia on next-gen and making accessible. Ex: GPUDirect over parallel SSD arrays is nuts.. and you can do it in python! Only really Google is competitive here (limited to ~TF), and maybe one day Apple.
Every indication from this report is they really do mean 2023, so why is Nvidia's fiscal year a full year ahead of the calendar year?
It's disconcerting to see operating expenses up 31% and income down 77%. I've never heard good things about Nvidia from employees, but as a consumer I've always rooted for them, being one of the few big names that seems to do honest to God innovative work pushing the boundaries of computing, as opposed to just milking user tracking to sell ads. It'd be nice to see this kind of business model work.
When I interviewed there, I was amazed at how much longer the tenure of my interviewers was than the 3 or 4 other big tech companies I’d interviewed at. I’m in the door now and still seeing the same thing. I think that speaks to employee satisfaction.
53 comments
[ 2.6 ms ] story [ 106 ms ] thread[1] Nvidia Quarterly Revenue Trend: https://s22.q4cdn.com/364334381/files/doc_financials/2023/Re...
On the off chance you get to put it in your cart, it either disappears by the time you check out or the order gets voided afterwards. It rarely stays in stock longer than a few mins.
Google is only giving me Allied Irish Banks. Somehow I feel that's not what you're talking about.
> Nvidia Add-in Board (AIB) partners ... The list includes Inno3D, MSI, ASUS, Palit, Colorful, PNY
https://old.reddit.com/r/Amd/comments/4pma4q/terminology_all...
Quote:
There seems to be some confusion regarding the acronym AIB, and many are using AIB to refer to 'non reference' graphics card designs.
AIB is an acronym for Add In Board as used within the video card industry. The 'graphics' part is implied by the industry context.
All Graphics cards are AIB.
http://jonpeddie.com/press-releases/details/add-in-board-mar...
An AIB supplier or an AIB partner is a company that buys the AMD (or Nvidia) Graphics Processor Unit to put on a board and then bring a complete and usable Graphics Card or AIB to market.
See AMD's article on Partners (including AIB, OE, System Builder) here: http://support.amd.com/en-us/kb-articles/Pages/AMDPartnersAI...
The term AIB has absolutely nothing to do with what ports are available, the design of the pcb, or the cooler design.
AIB literally just means "it's a graphics card".
Tech news people tend to use "AIBs" as shorthand for "second-party (add-in) board partners" -- that is, companies that sell graphics cards using AMD or Nvidia GPUs but with their own board and cooler designs. So in the case of Nvidia this would be EVGA, Gigabyte, MSI, Zotac, ASUS, etc. But crucially NOT including Nvidia themselves, even though Nvidia sell their own reference design they call the "Founder's Edition".
Try different countries - there will be delivery cost, but it will not be so comparable to the price.
That's phenomenal growth for a product category that was a very short time ago almost exclusively desktop-only. I guess ML workloads are huge and here to stay.
In a supposed down economy c. early next year, offering processor power and hard disk space from home machines as a source of passive income could be the next OnlyFans.
With disk space, you have the issue of liability—what happens when someone decides to store child pornography on my hard drive? That's a more solvable problem, but I'm certainly not envious of anyone who wants to tackle it.
The reason you can’t find 4090s is because Nvidia no longer cares about retail consumers.
I think so but it deserves a bit of a caveat: a lot of ML workloads right now are distinctively unprofitable, and like all of the unprofitable-venture-funded businesses of the past decade, many of which have been culled in the recent downturn, there will have to be a reckoning at some point.
There are definitely lots of workloads that produce positive ROI for companies, but many more that are heavily subsidized (ex. products like Alexa and Google Assistant) and consume a vast amount of ML resources.
A lot of ML-centric products suffer from a significant departure from the traditional Silicon Valley notion of each additional user being zero marginal cost, and products having negligible operating costs vs. high fixed costs.
As a researcher, I think we'll go into a "winter" soon, but nowhere near the previous one. The hype will die down but ML has shown to be extremely useful. Anything with graphic design, gaming, video, imaging, streaming, etc uses ML these days. It's been used in science for awhile (under statistical learning, e.g. regression) and we can't do without it. Though a lot of these things aren't following the main hype machine. But if you're in the weeds you'll see a lot of these works.
But I do think the hype will die. People even now are starting to notice that a lot of the diffusion work looks amazing at first glance but has major errors. There's also the issue of alignment. These are going to likely take a lot of mathematical experience to push forward[0] and will benefit less from a pure numbers game.
Of course, the other part (in Nvidia's favor) is that a lot of scientific work is GPU based now. Not just ML. Parallelism is king. You can do FEM, particle simulations, etc with GPUs now because the clock speed got high enough. So you can do "heavy" calculations with extreme parallelism. Previously you could only do this with lighter loads. (there's also major algorithmic improvements, so I don't want to undermine the fantastic work of those researchers).
[0] I want to admit my bias here. I'm a "math person" (not a mathematician, they will school me) and there's not a lot of ML researchers going down this path. To give an example, there are plenty of diffusion researchers I've talked to that don't know how to calculate the likelihood of their samples while they use an ELBO cost function. But this comes into the alignment issue because we need to talk about the smoothness of latent manifolds, interpolation, inversion (not just image inversion, but image/text inversion), and a lot more. I just don't think we can continue on this path of bigger networks, bigger data, search hyper-parameters, get benchmark result. (I fundamentally believe this is anti-scientific fwiw)
It’s not just the hardware like you’re talking about. The tooling has grown up insanely well too. You used to have to write CUDA or similar if you wanted a simulation on a GPU. Now you can write it in a high level language in a way that looks pretty close to the math.
My only caution on that is that I think their moat on ML is much shallower than people think. nVidia has traditionally maintained their position as the top dog in GPGPU by being first to provide good hardware and software, having markets lock in on that software, and then able to later sell more hardware at very good margins. Even when AMD (and others) later provide hardware that provides much better perf/cost for the same workload, at that point everyone already has a small mountain of tightly optimized cuda, and the cost/benefit of switching hardware providers just isn't there.
I really don't think this is going to work in ML. At their core, the ML algorithms are generally simple enough to fit on a blackboard, and are relatively much more easy to efficiently parallelize and optimize on diverse hardware. nVidia has again managed to capture most of a nascent market by being the first with adequate hardware and software to support it. But I think in a couple of years there are going to be a lot more competitors in this space, and unlike with previous GPGPU work, it will not be that painful for the end-user to shop around.
This won't mean that AMD (or someone else! this space is much easier to enter than GPUs in general) will capture the value from nVidia. Instead it will mean that the entire space will be commoditized and the margins will absolutely collapse.
I think the inertia of switching is lower for ML because libraries like pytorch support more backends besides just CUDA, such as ROCm (AMD) and MPS (Apple Metal). I used the pytorch ROCm backend recently and there's certainly some more work that needs to be done in this space to get it to the same level as the CUDA backend (performance and compatibility wise), but at least it makes other GPU vendors an option.
It's disconcerting to see operating expenses up 31% and income down 77%. I've never heard good things about Nvidia from employees, but as a consumer I've always rooted for them, being one of the few big names that seems to do honest to God innovative work pushing the boundaries of computing, as opposed to just milking user tracking to sell ads. It'd be nice to see this kind of business model work.
There's some impressive growth there. What kind of moat does the company have on this?
AMD posted 1.6b in gaming sales this Q, while Nvidia did 1.57b.
AMD closing in on Nvidia in terms of overall sales. Might see them past them up by the end of next year.