this kind of headline is like, Elon Musk started as a helpless infant who couldn't speak or raise his head under his own power and now he is a multi billionaire who buy twitter so everyone have to read his tweet
haha yes. tbh I don't get why everyone is so hysterical about nvidia right now like this article. It's a chip company. We knew chip companies are important ever since ic's were invented in 1959. The technology is fascinating but it's also pretty predictable, nothing prophetic here.
I admit to the extent I thought about nvidia I just kind of assumed they would fade away as companies like intel and apple integrated graphics processing into their own SOCs in phones tablets laptops etc.
And I also figured the crypto boom was a temporary unsustainable surge in the market for discrete gpus.
Now LLMs have taken up the crypto slack in terms of gpu demand. (I think? Is this what’s going on?) The question I have is whether there is any reason nvidia won’t be commoditized out of this market. Maybe someone who is smart at hardware can explain.
If China can learn to get along with everyone, and they are allowed to purchase high-tech manufacturing equipment needed to train a workforce over the next two decades, then yes, it will eventually be commoditized.
I think they basically have, it is just a really, really difficult problem to make a CPU that is competitive with Intel and a GPU that is competitive with NVIDIA, so putting it all together is not for the faint of heart.
I'm not sure this is obvious. For the consumer market, maybe.
But at the end of the day, there are physical limits on how much compute you can squeeze into any given area. If your job requires more parallel compute than can reasonably be squeezed into a package for power/thermal reasons, then distributing the workload off-package is a requirement.
Maybe hardware will morph into a more heterogenous mix than we have now, with many single-package cpu-gpu nodes working in parallel, instead of a few cpus orchestrating a gigantic sea of GPUs, but maybe not.
I'm more convinced that this is just one stage of the cycle. GPUs in consumer hardware weren't really a thing before the 90s. From the 90s to like 2013 graphics software (and it's ability to load hardware) improved rapidly. Since then it's kind of stagnated and a lot more of the focus has shifted to doing the same amount of rendering with less power/less heat/less space. Even if we do see a shift toward SoCs/APUs/whatever in consumer hardware over the next few years, I'd bet on some sort of paradigm shift to come along (fully raytraced rendering?) and swing the pendulum pack toward big discrete GPUs at some point again.
I think Musk bought Twitter to control the media, Murdoch style.
It makes a lot of sense, it makes even more sense to have 90% of the public believe you're an idiot who spent 50 billion dollars on a lark because you're childish, which mean's they're less untrusting of your media ambition.
It’s not survivorship bias. Your business must have a scrappy, relatable origin story. If someone with millions of dollars was involved, omit that part.
Except waffles. The strangest thing about waffle house is that there waffles are terrible. Every other option has better waffles. I like everything else at waffle house. Curious if others feel the same?
I understand this sort of articles are for entertainment and are meant to be relatable, but this one is pretty ridiculous, what does "started in a Denny's" even mean? did Nvidia's founder teach himself electrical engineering while he was waiting tables at Denny's or something. Next time I start something I'm only talking about it at el pollo loco.
Jensen Huang, Chris Malachowsky, and Curtis Priem were eating at a Denny's one night - because it's literally the only place open at 10 pm in the Bay Area - when Jensen bit onto a piece of silica sand that had made it into his Grand Slam Breakfast and very painfully chipped his tooth.
His yelled "GPUUUUUUUU" in pain, and NVIDIA was born. Instead of a bill, the waiter brought them the prefilled incorporation documents and on the way out the door, the hostess handed them their seed round.
Originally, Malachowsky said in an interview that he thought she was an agent of the Sand Hill Road cult, carefully placed at a known Silicon Valley nexus of innovation ley lines (again, because Denny's is the only place open late). However, after six months of exhaustive searching and $20 million spent on seances and psychics in preparation for IPO, several hundred Goldman Sachs and JP Morgan agents failed to locate her despite a dozen McKinsey partners sacrificing a truckload of silicon wafers at the fiscal solstice.
Some believe her to be an incarnation of Calafia, the ancient queen of California. Other suspect she is the personification of the pay-it-forward spirit of Silicon Valley. Others still believe she was a demon sent to destroy ATI or bring on the dark age of thinking machines.
I heard that Jensen actually saw one of the cooks take the fries out of the frier and remembered that the British called them chips, spurring an inspiration to create a new kind of chips.
This is a classic example of survivorship bias. These stories never mention the many stomachs that did not survive the Denny's Super Slam Ham & Five-Cheese Omelette.
I don't expect any single article to cover all the details of a company that just passed it's 30th year. And even then, they're missing bits of lore, like the CEO playing ping pong as a teenager (see https://engineering.stanford.edu/news/jen-hsun-huang-nvidia-...).
This WIRED article from 2002 gives an interesting snapshot of the same company and CEO:
Meet Nvidia CEO Jen-Hsun Huang, the man who plans to make the CPU obsolete.
https://www.wired.com/2002/07/nvidia/
- Quote: "What we can do in the next five years is going to blow your mind. In 10 years, we should be bigger than Intel." (which took until July 2020)
Then you can see how things evolved by the 2009/2010 timeframe, plus where they thought they might be headed:
Mercury News interview: Jen-Hsun Huang, co-founder of Nvidia
https://www.mercurynews.com/2009/10/09/mercury-news-interview-jen-hsun-huang-co-founder-of-nvidia/
I’m Prepared for Adversity. I Waited Tables.
https://archive.is/2iUbf
This was interesting to me as I sometimes wondered hth they came up with the name:
> At the time, graphics cards were known by two letters, and Priem liked the sound of “NV.” Envy. They called their startup Nvision, which wasn’t a real word and didn’t hint at what they would do. It was perfect. [...] The only problem was that several companies already had similar names, including one that made environmentally friendly toilet paper. Priem reached for his Latin dictionary and found the word for envy. Invidia. He dropped the first letter [...]
I recall my relatives asking me in … 98? 99? timeframe what they should invest some spare 10k or so cash in - 3dfx or Nvidia, and it was a clear choice to me - Voodoo3 was far superior to to anything budget-choice Nvidia had at a time. I wonder if they are reading this article.
3dfx was an incredible company but they made mistakes when it came to partners and they basically self destructed. One can only imagine what could have been.
Stories like these are what I would call "humility porn." So Huang succeeded partly because he worked as a waiter and taking "pancake orders" taught him how to deal with strangers? Do the authors realize how ridiculous this sounds?
That's great if Huang found a way to draw on his life experience. But he also just happened to have a master's degree from Stanford University and lived in Silicon Valley. Those are the sorts of characteristics that narrow down the possibilities quite a bit. Maybe he really did have a humble background. I don't know. But can we quit implying that's really what matters?
Curiously, a multitude of AI algorithms can figure which 'features' are relevant (ignoring training data requirement for a second*), to what extent, and how to use them as needed. On the other hand, humans (the authors in this example) often don't understand that, and can take any of the input features as the key determining factor.
* When, why, and if, AI needs more training data than humans aside, even humans have received enough of it to have developed some common sense, but yet ...
Regardless of how humble, good business strategy can be useful to pay attention to.
There was a recent posting of a twitter thread by Peter Yang discussing the $Trillion valuation.
I added a link to a different thread where he pointed out Nvidia CEO Jensen Huang gave an inspiring commencement speech on May 28 where it was briefly mentioned the way Nvidia almost failed on 3 different occasions:
For those who complain about the headline: if you listen to WSJ's recent podcast episode on Nvidia, parts of it are equality ridiculous and embarrassing.
...but of course it wouldn't matter to people who don't have the technical background.
I think the gripe people have with the title is insecurity. They feel that they're "less than" because they can't go from humble beginnings to being a master of the universe as the title here implies.
97 comments
[ 4.6 ms ] story [ 156 ms ] threadAlso, none of those other chip companies have broken US$ 1 trillion in market cap. That's a notable achievement.
To elevate a stock, a tulip, or a religious belief, to levels that far exceed any reason - that's just a part of being human.
And I also figured the crypto boom was a temporary unsustainable surge in the market for discrete gpus.
Now LLMs have taken up the crypto slack in terms of gpu demand. (I think? Is this what’s going on?) The question I have is whether there is any reason nvidia won’t be commoditized out of this market. Maybe someone who is smart at hardware can explain.
But it isn’t clear to me if Intel or Nvidia are further ahead when it comes to figuring that out.
Maybe hardware will morph into a more heterogenous mix than we have now, with many single-package cpu-gpu nodes working in parallel, instead of a few cpus orchestrating a gigantic sea of GPUs, but maybe not.
I'm more convinced that this is just one stage of the cycle. GPUs in consumer hardware weren't really a thing before the 90s. From the 90s to like 2013 graphics software (and it's ability to load hardware) improved rapidly. Since then it's kind of stagnated and a lot more of the focus has shifted to doing the same amount of rendering with less power/less heat/less space. Even if we do see a shift toward SoCs/APUs/whatever in consumer hardware over the next few years, I'd bet on some sort of paradigm shift to come along (fully raytraced rendering?) and swing the pendulum pack toward big discrete GPUs at some point again.
It's kind of amazing how perfect the timing was for LLM popularity (right around the time Ethereum moved to PoS, and the crypto market cooled down).
I’m not saying that makes Nvidia worth 1T with it’s expected profit.
It makes a lot of sense, it makes even more sense to have 90% of the public believe you're an idiot who spent 50 billion dollars on a lark because you're childish, which mean's they're less untrusting of your media ambition.
Now it's worth a $1T.
* Please do not respond that you haven't been to a Denny's. It is a figure of speech called hyperbole.
His yelled "GPUUUUUUUU" in pain, and NVIDIA was born. Instead of a bill, the waiter brought them the prefilled incorporation documents and on the way out the door, the hostess handed them their seed round.
Some believe her to be an incarnation of Calafia, the ancient queen of California. Other suspect she is the personification of the pay-it-forward spirit of Silicon Valley. Others still believe she was a demon sent to destroy ATI or bring on the dark age of thinking machines.
We will never know.
I got to meet the fish Friar and the chip Monk.
I don't expect any single article to cover all the details of a company that just passed it's 30th year. And even then, they're missing bits of lore, like the CEO playing ping pong as a teenager (see https://engineering.stanford.edu/news/jen-hsun-huang-nvidia-...).
This WIRED article from 2002 gives an interesting snapshot of the same company and CEO:
Then you can see how things evolved by the 2009/2010 timeframe, plus where they thought they might be headed:> At the time, graphics cards were known by two letters, and Priem liked the sound of “NV.” Envy. They called their startup Nvision, which wasn’t a real word and didn’t hint at what they would do. It was perfect. [...] The only problem was that several companies already had similar names, including one that made environmentally friendly toilet paper. Priem reached for his Latin dictionary and found the word for envy. Invidia. He dropped the first letter [...]
That's great if Huang found a way to draw on his life experience. But he also just happened to have a master's degree from Stanford University and lived in Silicon Valley. Those are the sorts of characteristics that narrow down the possibilities quite a bit. Maybe he really did have a humble background. I don't know. But can we quit implying that's really what matters?
* When, why, and if, AI needs more training data than humans aside, even humans have received enough of it to have developed some common sense, but yet ...
There was a recent posting of a twitter thread by Peter Yang discussing the $Trillion valuation.
I added a link to a different thread where he pointed out Nvidia CEO Jensen Huang gave an inspiring commencement speech on May 28 where it was briefly mentioned the way Nvidia almost failed on 3 different occasions:
https://twitter.com/petergyang/status/1662831418882560000
At the time this did not appear to be appreciated at all.
Maybe this far into June there will be others who find his recollection worthwhile.
As an excerpt:
"Run don't walk. Either you're running for food, or you are running from being food:
1. Have the humility to confront failure and ask help.
2. Endure the pain needed to realize your dream.
3. Make sacrifices for your life's work."
...but of course it wouldn't matter to people who don't have the technical background.
Before the pinto.