Here's a question since everyone keeps bringing up the DotCom bubble. Although the bubble burst, have not the people who were building in the 1999, still more than made up for their losses by having the know-how and being able to capitalize on the subsequent emergence of the WWW as we know it today?
And that current valuation of the underlying market is x times the real valuation. And real valuation can be significantly non-zero. Take housing market. Most of the houses do have some real value. And even if the bubble pops, it doesn't take their values to zero, it might be momentarily be just below real value.
I look at it as investor error due to lack of expertise in the business area. If investors understood the economics of internet businesses better (and almost nobody did in something this new) they may have helped grow the Internet economy even faster than it did, without the bubble or burst.
Likewise. And I look at the current situation and also see investor error, but in the opposite direction, as a great many investors equate LLMs to AI/ML, and vastly underestimate the space in which this technology will disrupt. I keep reading pieces in Forbes and similar which are like “does the world really need a better chatbot?” or “can Google justify spending $10bn on making a better cat picture?”
Outside of LLM's, which AI/ML product has the potential for widespread consumer adoption? I recognize the usefulness of modern AI in all sorts of domains, but they are almost entirely niche services for business and not targeted to the broad consumer. The problem then is not that AI is not cool or useful, just that it won't make as much money as the enthusiasts think, at least not now.
Those “niche services” are potentially much more lucrative than consumer-facing products. For instance, an ML quant could be a license to print money, and would have appropriate costs. Designing more efficient (cheaper) structures and machines doing FEA/CFD better than current algorithmic models would be a huge boon to engineering industries. Sentiment analysis and forecasting for government. Military strategy that outsmarts any human strategist. Demand lead pricing for products and services. JIT manufacturing management through superior cost and demand forecasting. Identification of baroque tax fraud. Diagnosis of medical conditions. Product recommendations. Tailored drugs. Pre-crime.
On it goes. These are just a few examples of where ML is already beginning to be applied and reaping rewards.
I think the bubble was more about the lack of viable business models. The technology was real and the investors saw it, but the business models needed another decade to mature.
The same could happen again. It may be easier to find good uses for AI than to make large amounts of money with it.
Follow up question: Where in the dotcom bubble is AI in 2024?
Is it already 2000? Or is it in the beginning, 1995? 1998?
For some context, if you invested in Nasdaq in Jan of 1995 and did not sell until September 2001, you'd still be up by 86%. And if you invested at the absolute peak of the bubble, you'd still be up 250% in 2024.
> Today, if you have a powerful enough GPU cluster, you can run an internal GPT4-level LLM by deploying Meta's LLama 3.1 405b.
You also have to be willing to consume ungodly amounts of energy to run those GPUs. That seems like an important caveat while the conversation about climate change and unpredictable weather is still top of mind for so many people.
I gather what the parent is saying is that it’s better to just keep investing in index funds or equivalents, because it’s impossible to tell where we are, i.e. timing the market.
I like to compare GPT3 to the 286 Processor and GPT4 to the Pentium chip.
There is still a lot that doesn't work but basic tasks can be done.
Right now most genAI applications are toys. And a lot still doesn't work text in Stable Diffusion, true reasoning/planning, agents with agency. A lot of demos and evals are optimized for marketing benefit but fail in production systems.
Gates Law variant: "People always overestimate the impact of a new technology in a year and underestimate its impact in ten years."
In 1999 people predicted that the Internet would change everything, in 2000 people called the Internet a flop and made fun of pets.com sock puppets, by 2010 the Internet had in fact changed everything.
We are more than 10 years in to crypto, and if it all vanished tomorrow, I can’t imagine anything changing other than a few indexes invested in it crashing. And less ransomware I guess.
Not sure the comparison is justified. Sure, AI is cool, hot tech, but the internet enables connection between virtually every single human alive. Completely different scale of influence.
You can shoot yourself in the foot and claim you became a better marksman in the process, that doesn't really do much to justify it though.
All other things being equal they'd been better off investing their money or time into something that wasn't a bubble economy. This is basically broken window logic.
Very different things. I lived through the dot com bubble (didn't get rich because I was busy doing a PhD instead). Basically, a massive amount of clueless idiots funding companies that were literally nothing more than a crappy website. It was over in a few years. At the peak of the hype, disgusting amounts of money got spent on companies that went absolutely nowhere because there was absolutely nothing there. And then it all fizzled out. But there was also a healthy amount of experimentation and new stuff happening.
This feels different; there's actually some substance to the madness. Quite a few of the companies being funded are actually creating some pretty cool tech. And there's some real revenue potential as well; it's not just investment money keeping everything going. A good dot com era company reference would be companies like Google or Amazon that took the cash and got a lot of that tech making money for them even after the investment bubble burst. They also grabbed some of the smarter people at the same time. There are a few more examples. If you squint a little, you can see a few companies that are likely to be able to start raking in lots of cash soon that are at this point well funded.
Also a lot of the current investment money is being converted into GPU hardware. Which is of course nice for companies like NVidia, whom are probably a bit over valued currently. But the point is that hardware is tangible. Even if the companies that buy it go bust, the hardware just ends up in the hands of others. We're talking many millions of GPUs that are being deployed and that, like it or not, will be doing a lot of AI workloads for years to come for whomever ends up owning it. And there are a lot of smart people trying to make that hardware do all sorts of cool stuff. Hardware is a much better asset to have than useless websites. And I don't think a lot of the software is that bad either.
Add that in the dotcom bubble, internet-flavored growth had never happened before, so it was relatively harder to predict the growth of (say) Google.
But today, deep-learning-flavored growth is a 10-year-old concept, and LLMs largely have leverage over existing (versus quite new) business models. There will probably be fewer new monopolies versus the dotcom era; in particular OpenAI has lots of competition.
I think a likely and hidden contributor to the BigTech layoffs of 2023 is the development of AI. I know there are other reasons, but it didn't hurt that they had labor saving AI systems built that could handle the load.
After hiring a lot of people, having systems built, and training data developed, they could replace large amounts of staff that used to need to do manual work those AI systems. Less manual moderation, more automated responses. It became "fashionable" to cut staffing levels, so companies followed suit.
Most tech leaders avoid talking about how much labor AI will replace. We don't know yet, but it likely follow trends of making junior roles harder to find while making seniors more valuable with a slight reduction in the overall workforce. This has been present in all industries
I don't agree with this take, the layoffs are companies simply returning to their regular baseline numbers. AI was just a comfortable scapegoat for many to put the blame on rather than "we like money".
Looking at the workforce and asking do these people produce enough value for their pay? Do they actually provide some new innovations that would justify their costs?
It is the same everyone does. It is not like these people will keep their plumber of home builder employed out of good will after the job is done. Or nanny around after their kids have fully grown...
I can see hype go through the roof again if someone like OpenAI delivers a model that is as big of a leap as GPT3 to GPT4. Other LLMs have caught up to GPT4, but surely OpenAI has been using the 2 year lead they had and readying GPT5?
There are two AI developments that I'm quite excited for:
1. Very large foundational models (such as GPT5-level LLMs)
2. Optimizations for smaller models, context size, inference speed, and multimodal LLMs
I think 2024 has been the year for #2, which is why the hype has died down a little. No splash big models that shock the world and make white collar workers shiver. But #2 is crucial to actually deploying LLMs to the masses.
GPT-4 was released in March 2023, one and a half years ago (I point that out because it feels like a different era). Not sure when they finished training it but I'm pretty sure it has nothing to do with (and it's in fact much later than) the training data cutoff, which was in September 2021.
> There are two AI developments that I'm quite excited for
What about these things excite you?
Just the tech hubris of growth at all costs or the hope of some use case that has yet to manifest?
> 1. Very large foundational models (such as GPT5-level LLMs)
Has work been announced on or a release date been set for a GPT5 or are you just excited it might happen? Why stop there? Go ahead, be excited about MilleniumGPT.
Is there evidence that bigger is better forever?
> 2. Optimizations for smaller models, context size, inference speed, and multimodal LLMs
To do what exactly?
Write fan letters to olympians for me?
Replace all of the real human friendships in my life?
To write an annoyingly verbose email from a bullet list that the recipient will be loathe to read and so will feed it into another agent to turn the email back into the same bullet list?
(even if you know LLMs are the future, please try to...)
Take the pessimistic view on LLMs for a moment. That this doesn't really work out and the expensive computers are a somewhat embarrassing misstep.
In that world, at least a few companies will find themselves with multibillion dollar high performance computers running on site. With capex and power numbers to scare the accountants and financial analysts. Not running LLMs.
That's a really fast computer. Computers can do stuff. If not LLMs, it's going to do other things.
To confidently sit out this capability acquisition round you need to be sure that LLMs are grossly overrated and also that all the competition will fail to find anything else to do with their GPU supercomputers. Oh, and that your own staff would also fail to find anything (else) useful to do with one.
I am completely comfortable expecting datacenter scale supercomputers with previously unimaginable compute to do interesting things. I wouldn't want to be the megacorp missing the revolution because I spent the money on dividends instead.
The price of these computers is high now, due to supply constraints, so buying now isnt necessarily the best time for this outcome. You can buy some later...
The things these can do is (a) memory bandwidth, Nvidia GPUs have been described as "the worlds most expensive memory controller". Generally very useful anywhere. (b) fast low precision floating point. Historically most of the uses of supercomputers wanted high precision, eg for physics etc but people are now experimenting with using lower precision given availability (c) fast networking. generally useful.
We were seeing a spread of GPU into other applications pre AI, like GPU driven databases, I think this stopped a bit just because of pricing and availability.
Lots of memory bandwidth, low latency high bandwidth connections between nodes, lots of floating point. The MI300 looking a bit like HPC being repurposed for AI is probably a win in terms of using the hardware for other things in the future.
I've had to double check the low precision point qualifier. I expected MI300X to run f32 and f64 at the same speed (and it does) and nvidia to have slower f64 (which it does). It looks like H100 is 250TF at half, 60TF at float, 30TF at double. MI300X is 650TF at half, 80TF at float or double. Very fast F16 indeed.
Or they invested in GPU’s to mine bitcoin and the ASIC’s are just starting to arrive. At least for inference, GPU’s are not the end game. And who knows if training will continue to need them.
If so there could be another round of investment at a better performance per watt, and specialised hardware will be the key. They’ll all be building custom chips.
The specialised ASIC approach is less likely to be useful for other things though, whereas GPUs are starting to threaten to run arbitrary software well.
Sure. My base case is that LLM’s find uses and that compute gets cheaper and more efficient, which helps make more business cases possible. Investing in current state GPU’s doesn’t seem that useful in that world. Nvidia is priced for a monopoly and I’m not sure it is one.
Imagine if LLMs didn't exist, tech companies hadn't massively invested in compute and money was put elsewhere, perhaps dividends.
In that world, a tech company could arbitrarily invest in multibillion dollar high performance computers for no particular reason. Following the argument, with all that compute on site with nothing to do engineers would would find something interesting and all other companies that didn't invest would be missing out on that revolution.
So by the argument, any tech company could gain a competitive advantage by any non-strategic investment on the assumption that it will always work out. But of course, this only works if you have infinite money and infinite opportunity to speculate. As soon as your resources are constrained, then strategic choice becomes the dominant factor and your CEO has a lot of explaining to do.
What we have here though is the fear of being the company that didn't make that investment while your peers did. That has a completely different risk schedule on it.
If it's a mistake, your excuse is that your competition did it too and you wanted to guard against being left behind. If it works, everyone is happy.
If you're the only company making that speculative investment, it's great if it works and you might be fired by the board if it doesn't.
But the risk/reward is the same regardless of the frame of reference. It's the difference in the perceived risk between gaining $10 and not loosing $10, even when the odds are the same. And we know that while rationally its the same, _loss aversion_ is a cognitive bias that makes people behave differently in these two cases. But FOMO is something we should be mitigating against, since our fear alone doesn't change the actual odds.
But I will take your point that you have more cover for making a bad decision if you know all your competitors are making the same bad decision. But you still missed the opportunity to not make a bad decision and therefore get ahead. The actual risk remains the same.
Funnily enough it kind of isn't. From the perspective of the company itself, it's performance relative to the competition that matters. Not buying the machine when others do is a win if the machines don't work out.
From the perspective of the people running a successful company, it's much more important to make easily defensible reasonable decisions than to make ambitious ones. You protect the capital in preference to maximising returns. Major gains are somewhat rewarded and major failures severely punished, with a comfortable baseline if you maintain the current positioning.
If you're not yet a successful megacorp, all the dials are turned in favour of risk because you need the reward. Lots of incomers doing riskier things seeking to overthrow the incumbent is roughly how we get a turnover of companies and a degree of overall progress.
I think this round is interesting because the incumbents have seen substantial competitive risk which could otherthrow them on an alarmingly short timescale (i.e. while the current leadership are still there), and that has induced otherwise fairly unlikely massive capex spend.
I guess if you are a multibillion dollar company, you can keep that in the bank (stock buybacks or whatever) or invest. So the question isn't really if AI is overhyped or not at the moment. The question is really do you know anything better to do with all that money to make your company future-proof.
55 comments
[ 5.1 ms ] story [ 107 ms ] threadIt means there's a mass over investment and over spend in companies trying to serve the demand at the time.
On it goes. These are just a few examples of where ML is already beginning to be applied and reaping rewards.
The same could happen again. It may be easier to find good uses for AI than to make large amounts of money with it.
Is it already 2000? Or is it in the beginning, 1995? 1998?
For some context, if you invested in Nasdaq in Jan of 1995 and did not sell until September 2001, you'd still be up by 86%. And if you invested at the absolute peak of the bubble, you'd still be up 250% in 2024.
Perhaps if you invested in Amazon or MS.
Meta, Anthropic, Google, Mixtral have demonstrated that OpenAI can be caught and matched.
Today, if you have a powerful enough GPU cluster, you can run an internal GPT4-level LLM by deploying Meta's LLama 3.1 405b.
If OpenAI's GPT5 is as big of a leap as GPT3 to GPT4, the hype will reach unprecedented level again in my opinion.
You also have to be willing to consume ungodly amounts of energy to run those GPUs. That seems like an important caveat while the conversation about climate change and unpredictable weather is still top of mind for so many people.
In 1999 people predicted that the Internet would change everything, in 2000 people called the Internet a flop and made fun of pets.com sock puppets, by 2010 the Internet had in fact changed everything.
All other things being equal they'd been better off investing their money or time into something that wasn't a bubble economy. This is basically broken window logic.
This feels different; there's actually some substance to the madness. Quite a few of the companies being funded are actually creating some pretty cool tech. And there's some real revenue potential as well; it's not just investment money keeping everything going. A good dot com era company reference would be companies like Google or Amazon that took the cash and got a lot of that tech making money for them even after the investment bubble burst. They also grabbed some of the smarter people at the same time. There are a few more examples. If you squint a little, you can see a few companies that are likely to be able to start raking in lots of cash soon that are at this point well funded.
Also a lot of the current investment money is being converted into GPU hardware. Which is of course nice for companies like NVidia, whom are probably a bit over valued currently. But the point is that hardware is tangible. Even if the companies that buy it go bust, the hardware just ends up in the hands of others. We're talking many millions of GPUs that are being deployed and that, like it or not, will be doing a lot of AI workloads for years to come for whomever ends up owning it. And there are a lot of smart people trying to make that hardware do all sorts of cool stuff. Hardware is a much better asset to have than useless websites. And I don't think a lot of the software is that bad either.
But today, deep-learning-flavored growth is a 10-year-old concept, and LLMs largely have leverage over existing (versus quite new) business models. There will probably be fewer new monopolies versus the dotcom era; in particular OpenAI has lots of competition.
After hiring a lot of people, having systems built, and training data developed, they could replace large amounts of staff that used to need to do manual work those AI systems. Less manual moderation, more automated responses. It became "fashionable" to cut staffing levels, so companies followed suit.
Most tech leaders avoid talking about how much labor AI will replace. We don't know yet, but it likely follow trends of making junior roles harder to find while making seniors more valuable with a slight reduction in the overall workforce. This has been present in all industries
I think it is:
1. Over hiring from covid
2. A switch from developing traditional software to one that is LLM based
The interest rates mean that the now gets a higher priority, and software ready in 2030 or later a lower priority.
It is the same everyone does. It is not like these people will keep their plumber of home builder employed out of good will after the job is done. Or nanny around after their kids have fully grown...
There are two AI developments that I'm quite excited for:
1. Very large foundational models (such as GPT5-level LLMs)
2. Optimizations for smaller models, context size, inference speed, and multimodal LLMs
I think 2024 has been the year for #2, which is why the hype has died down a little. No splash big models that shock the world and make white collar workers shiver. But #2 is crucial to actually deploying LLMs to the masses.
What about these things excite you?
Just the tech hubris of growth at all costs or the hope of some use case that has yet to manifest?
> 1. Very large foundational models (such as GPT5-level LLMs)
Has work been announced on or a release date been set for a GPT5 or are you just excited it might happen? Why stop there? Go ahead, be excited about MilleniumGPT.
Is there evidence that bigger is better forever?
> 2. Optimizations for smaller models, context size, inference speed, and multimodal LLMs
To do what exactly?
Write fan letters to olympians for me?
Replace all of the real human friendships in my life?
To write an annoyingly verbose email from a bullet list that the recipient will be loathe to read and so will feed it into another agent to turn the email back into the same bullet list?
That they will annoy the hell out of you. ;)
Take the pessimistic view on LLMs for a moment. That this doesn't really work out and the expensive computers are a somewhat embarrassing misstep.
In that world, at least a few companies will find themselves with multibillion dollar high performance computers running on site. With capex and power numbers to scare the accountants and financial analysts. Not running LLMs.
That's a really fast computer. Computers can do stuff. If not LLMs, it's going to do other things.
To confidently sit out this capability acquisition round you need to be sure that LLMs are grossly overrated and also that all the competition will fail to find anything else to do with their GPU supercomputers. Oh, and that your own staff would also fail to find anything (else) useful to do with one.
I am completely comfortable expecting datacenter scale supercomputers with previously unimaginable compute to do interesting things. I wouldn't want to be the megacorp missing the revolution because I spent the money on dividends instead.
The things these can do is (a) memory bandwidth, Nvidia GPUs have been described as "the worlds most expensive memory controller". Generally very useful anywhere. (b) fast low precision floating point. Historically most of the uses of supercomputers wanted high precision, eg for physics etc but people are now experimenting with using lower precision given availability (c) fast networking. generally useful.
We were seeing a spread of GPU into other applications pre AI, like GPU driven databases, I think this stopped a bit just because of pricing and availability.
I've had to double check the low precision point qualifier. I expected MI300X to run f32 and f64 at the same speed (and it does) and nvidia to have slower f64 (which it does). It looks like H100 is 250TF at half, 60TF at float, 30TF at double. MI300X is 650TF at half, 80TF at float or double. Very fast F16 indeed.
(sourced from https://www.techpowerup.com/gpu-specs/radeon-instinct-mi300x... and https://www.techpowerup.com/gpu-specs/h100-sxm5-96-gb.c3974, I might have the wrong SKUs for comparison)
If so there could be another round of investment at a better performance per watt, and specialised hardware will be the key. They’ll all be building custom chips.
In that world, a tech company could arbitrarily invest in multibillion dollar high performance computers for no particular reason. Following the argument, with all that compute on site with nothing to do engineers would would find something interesting and all other companies that didn't invest would be missing out on that revolution.
So by the argument, any tech company could gain a competitive advantage by any non-strategic investment on the assumption that it will always work out. But of course, this only works if you have infinite money and infinite opportunity to speculate. As soon as your resources are constrained, then strategic choice becomes the dominant factor and your CEO has a lot of explaining to do.
If it's a mistake, your excuse is that your competition did it too and you wanted to guard against being left behind. If it works, everyone is happy.
If you're the only company making that speculative investment, it's great if it works and you might be fired by the board if it doesn't.
But I will take your point that you have more cover for making a bad decision if you know all your competitors are making the same bad decision. But you still missed the opportunity to not make a bad decision and therefore get ahead. The actual risk remains the same.
From the perspective of the people running a successful company, it's much more important to make easily defensible reasonable decisions than to make ambitious ones. You protect the capital in preference to maximising returns. Major gains are somewhat rewarded and major failures severely punished, with a comfortable baseline if you maintain the current positioning.
If you're not yet a successful megacorp, all the dials are turned in favour of risk because you need the reward. Lots of incomers doing riskier things seeking to overthrow the incumbent is roughly how we get a turnover of companies and a degree of overall progress.
I think this round is interesting because the incumbents have seen substantial competitive risk which could otherthrow them on an alarmingly short timescale (i.e. while the current leadership are still there), and that has induced otherwise fairly unlikely massive capex spend.
1. No one would say the eye is conscious and yet its pattern matching is being re-used
2. The ML in the human brain re-uses the eye-cell wiring to pattern match....yes their was even a post either Friday or Sat here about that...
If we then re-tool the under-pining of AI, i.e. ML to match the new discovery we still get not AI but a new ML tool...
Or in short words find a fund index that is betting against the AI hype as the explosion will be massive.