This isn't new, it was announced a while ago. The A800 is a variant of the previous generation A100 chip which Nvidia cooked up specifically to sneak under the sanctions imposed on China, but with the US cracking down further they are now cutting their losses and selling them off in other markets.
Well the US did follow up by also banning the exports of the A800/H800 models tailored to precisely comply with the earlier sanctions, so presumably the intent was to stop export of GA100/GH100 silicon altogether and they didn't anticipate Nvidia rules-lawyering the sanctions by reconfiguring it.
I mean fuck around and find out. Folks want to slide right up to the edge of legal definitions that's fine, but playing victim when the powers that be see you do it and nudge the line closer to the spirit of the original intent is laughable. It's not like the concept of legislation was invented two weeks ago and folks are shambling around trying to figure out what it all means.
It's just frustrating for me to see people write rules, and then say "hey not like that!". It wastes everyone's time, the point of regulations is to draw the line and have no ambiguous meaning.
It's like putting up a speed limit sign that says "55mph" and then ticketing them for speeding going "53mph".
If they want people to go 35mph, they should just made the limit 35mph!
Consider that regulations are hard to get right and all the people responsible might not have known in advance all the edge cases that were possible. After all, if they knew it in advance, the regulation would have been made before it ever became a problem in the first place
This is one of those situations where how the government works is not like your personal life experiences. Nvidia wasn’t some anonymous individual out of hundreds of millions subjected to a capricious law. They were the direct target of the law, and almost certainly had a seat at the table when it was written. I’m sure it wasn’t what they wanted, but they certainly had to know what the intent was, including backroom, private conversations.
They thought the government wouldn’t dare stop them. The government called their bluff.
I sympathize with your position but have you ever participated in any activity where large numbers of relatively intelligent people are incentivized to probe the boundaries of "permissible" behavior? Video game design and development would take a tenth of the effort if corralling users away from game-breaking edge cases wasn't required, and that's a zero risk low reward endeavor. Now just imagine what those same folks would get up to if there was a few billion in loose cash thrown into the mix. All of that is to say defining the boundaries of acceptable behavior precisely is easily one of the hardest things to get right, and usually impossible to get right the first time.
Eh, there's some reason to be annoyed here. They would have banned GPU export entirely, I'm sure, except for industry lobbyists who said that doing so would make Chinese companies fill the void in offering GPUs for the lucrative Chinese gaming market, then it'd be a short step from there to having created the capability they rely on the west for. NVIDIA, AMD, Nintendo, Sony, Microsoft, etc. all lobbied for this outcome saying, essentially, "let us keep the gaming market, but just outlaw export of GPUs for AI training: that will be more effective because then there's no market for a local competitor to serve."
Then they went and made AI-trining equipment anyway that skirted the letter of the law.
I don't disagree with you but this is still very much on NVidia. They have the resources to contact the government and ask, "what if?" but they didn't because they were trying to pull a fast one.
NVidia knew that was a possibility when they were working around the intent of the trade restrictions. The Department of Commerce was quite transparent in their intentions, and NVidia undermined that and the obvious happened.
That's not what the US wants. The US wants to sell to China but not boost their military.
> U.S. Commerce Secretary Gina Raimondo, speaking in an interview with Reuters on Monday, said Nvidia "can, will and should sell AI chips to China because most AI chips will be for commercial applications."
Gina Raimondo thinks it's sneaking
> "That's not productive," Raimondo said. "I am telling you if you redesign a chip around a particular cutline that enables them to do AI, I am going to control it the very next day."
Not particularly as you're ignoring the environment when the rules were made. When the rules were made they made them as constrained (we want that right?). So the remaining chips were the ones nvidia had two/three product segments down. Then nvidia introduced a new chip that was 99% instead of just selling the ones they had that were 75/80%. You can see how nvidia is participating in bad faith over this. Blaming the g-men for releasing constrained, market-aware rules is itself a bad faith participation.
IMO, regulation is supposed to be “we think this is the line that makes chip too dangerous”. And nvidia is complying with that. If the danger line is 80%, why not set the line at 81%
The administration feels like it’s throwing a tantrum about an extra 19% of performance. The fact that that even matters, or the premise that chip is dangerous, are both just funny (not so fun for nvidia)
If there’s a line that actually makes product dangerous, the government should set it there. Designing laws they just-so happen to only hit specific SKUs seems like the original bad-faith act.
I don't get it, the rules were "Do not cross this line", nvidia, being a business that makes products that comply with regulations, made a product that stopped just short of the line.
It's like putting up a speed limit sign that says "55mph" and then ticketing them for speeding going "53mph"
Nvidia clearly attempted to circumvent the _spirit_ of the law. The intent of the ban was clear and Nvidia ignored that.
It's much less like the speed limit example you gave and much more like a parent telling one of their children to stop touching the sibling so they just hover an inch away from them saying, "I'm not touching you!". The parent is going to very obviously modify their rule immediately to curb this circumvention.
If I had to guess, Nvidia hoped the regulations were just signalling rather than an actual attempt to prevent the specific hardware from getting to the Chinese.
It's not like that at all. It's more like "don't sell F22s to anyone else" then Lockheed makes a slightly slower and less stealthy export model to skirt the rules. It's obvious that Nvidia is acting in bad faith, and it's funny that the same people who said we need to maintain an edge over China are probably defending Nvidia doing this.
I don't really care about what exactly they said, it's obvious to anyone with a brain that the intention of this restriction is to keep China from getting the highest end chips for AI development and putting a chip, that didn't exist when the regulation was written, right up against the restriction was obviously bad faith on Nvidia's part.
They're referring to the spirit of the law versus the letter of the law. NVidia knew they were not following the spirit of the law, and they understood that a result might be that the restrictions would simply be updated if they tried to undermine it.
Are higher numbers better or lower numbers better? A6000? A100? A800?
And H > A > V > K, I think? And there's no T100 for Tesla and J100 for Jetson? But there's a Tesla V100 (Tesla Volta 100) and a Jetson Xavier which is neither Jetson nor Xavier architecture but Volta architecture?
Can we please just have monotonically increasing product names in terms of compute capacity and stop jumping around the alphabet?
I know they name architectures after scientists, that's cool, but at least they could do what Ubuntu does with release names and go in alphabetical order.
It's even worse than that. L > H > A, but that's for the data center cards. If you want the latest Ada Lovelace core in a workstation card, that's an RTX 4000 Ada Edition series (and it does not use the old Quadro naming). This is different from a Quadro RTX 4000 - that's two generations behind, a Turing. An RTX A4000 is one generation back, an Ampere. Oh, and gaming cards which may give you better performance for less are mashed into the same numbering scheme. An RTX 4070 is a gaming card, and it exist in vanilla, Super, Ti, and Ti Super flavors. And what makes a card "gaming" vs "workstation"? The first have a gaudier-looking heatsink and less memory. Makes sense, right?
Care to explain why? The Orin option seems massively more scalable and you can put in as beefy a machine as you want without paying the Apple tax which scales exponentially with machine specs.
You will be able to resell that 192G Mac Studio for 3-4k after a few years easily, which you've bought for 6k. There's nothing competing with Apple Silicon right now for local LLMs when you look at TCO.
I don't understand anything you're saying. Orin is not scalable at all and (IMO) overpriced given its slow CPU and GPU and it's limited to 64GB of soldered RAM while Macs can go up to 192GB. AFAIK Orin is limited to a special build of Ubuntu and has no community.
> Orin is limited to a special build of Ubuntu and has no community.
Not anymore, now they allow a list of distributions. It has NVidia's first class support with all things optimized. Like CUDA, Pytorch, and so on. For now it's the best thing money can buy for mobile robot. I'm interested in Pytorch, CNNs, LLMs, video processing sort of things.
Community is nice, but not sure it's possible to get reasonable performance easily from Mac Studio. Don't remember seeing Apple in the list of supported platforms for PyTorch. Nothing personal, I'm interested in the results, not the process.
Generally, is there a resource that has a table of how much VRAM each of the open source models require, especially at different quantization levels? I am pretty confused about these Mixture of Experts models like Mixtral 8x7B.
I recognize that these massive memory GPUs are for "AI" applications, but it really is amazing (exponential growth, I know, I know) when I think back to being at uni when I was super excited to get a Real(tm) High performance(r) GPU that had a truly massive 16Mb of ram :D (and a whopping 64Mb of system ram!)
53 comments
[ 2.7 ms ] story [ 116 ms ] threadAnd for some people -- though not me -- it is actually pocket money and less than a weekend in Vegas.
Like Ferraris, houses in Pebble Beach, and many many other things.
Modulo the fact that people will spend $15k customizing a Honda Civic.
If the US did not want NV to sell any GPU to any Chinese company, they could make that the law instead
They should have just written this in the regulations.
Instead they're making nvidia waste resources making a product that complied with regulations.
It's like putting up a speed limit sign that says "55mph" and then ticketing them for speeding going "53mph".
If they want people to go 35mph, they should just made the limit 35mph!
They thought the government wouldn’t dare stop them. The government called their bluff.
Then they went and made AI-trining equipment anyway that skirted the letter of the law.
> U.S. Commerce Secretary Gina Raimondo, speaking in an interview with Reuters on Monday, said Nvidia "can, will and should sell AI chips to China because most AI chips will be for commercial applications."
Gina Raimondo thinks it's sneaking
> "That's not productive," Raimondo said. "I am telling you if you redesign a chip around a particular cutline that enables them to do AI, I am going to control it the very next day."
The administration feels like it’s throwing a tantrum about an extra 19% of performance. The fact that that even matters, or the premise that chip is dangerous, are both just funny (not so fun for nvidia)
It's like putting up a speed limit sign that says "55mph" and then ticketing them for speeding going "53mph"
It's much less like the speed limit example you gave and much more like a parent telling one of their children to stop touching the sibling so they just hover an inch away from them saying, "I'm not touching you!". The parent is going to very obviously modify their rule immediately to curb this circumvention.
If I had to guess, Nvidia hoped the regulations were just signalling rather than an actual attempt to prevent the specific hardware from getting to the Chinese.
What they actually said (abbreviated) was anything with more then
> "600 Gbyte/s [memory transfer speed]" or 4800TOPS
Page 103 https://public-inspection.federalregister.gov/2022-21658.pdf
They set very specific performance restrictions, not a general "don't sell fast stuff plz"
Are higher numbers better or lower numbers better? A6000? A100? A800?
And H > A > V > K, I think? And there's no T100 for Tesla and J100 for Jetson? But there's a Tesla V100 (Tesla Volta 100) and a Jetson Xavier which is neither Jetson nor Xavier architecture but Volta architecture?
Can we please just have monotonically increasing product names in terms of compute capacity and stop jumping around the alphabet?
I know they name architectures after scientists, that's cool, but at least they could do what Ubuntu does with release names and go in alphabetical order.
(1) https://www.nvidia.com/en-au/design-visualization/desktop-gr...
As per: http://web.archive.org/web/20231103222724/https://www.nvidia...
that said, i agree with your question!
Not anymore, now they allow a list of distributions. It has NVidia's first class support with all things optimized. Like CUDA, Pytorch, and so on. For now it's the best thing money can buy for mobile robot. I'm interested in Pytorch, CNNs, LLMs, video processing sort of things.
Community is nice, but not sure it's possible to get reasonable performance easily from Mac Studio. Don't remember seeing Apple in the list of supported platforms for PyTorch. Nothing personal, I'm interested in the results, not the process.
Correcting myself, it's there. Which makes Mac Studio an interesting option.
Generally, is there a resource that has a table of how much VRAM each of the open source models require, especially at different quantization levels? I am pretty confused about these Mixture of Experts models like Mixtral 8x7B.