Meta's AR/VR strategy is still the over-arching priority and given how well Quest 3 and Ray Bans glasses are doing shows that it is likely the right one. Especially once Apple Vision Pro ignites the industry.
AI will be used to enhance that e.g. generated avatars, autonomous agents, hand/body tracking etc.
Strong adoption by the younger TikTok generation which is an area that Meta has been desperate to bring back into their fold.
Also the product is pretty impressive. The camera quality is really good and the AI features genuinely useful. Definitely caused many in the industry to wonder if that form factor could be the future of the AR industry in the short term as well as the long term.
> Especially once Apple Vision Pro ignites the industry.
I may be repeating my own errors with the following as I said much the same for the iPhone and the Apple Watch when they were new, but…
The prices Apple are asking for seem excessive given what the products actually do, surely the cheaper alternatives are going to be what really matters?
(In this case, cheaper alternatives would include the Meta headsets).
Watched a guy spend $6K on iPads for his kids like it was a stocking stuffer. Took 15 minutes as he had to call the wife on if 1TB was enough. Felt bad I was just buying an Air so wandered to let the sales kid work him.
You've just reminded me how weird I am with money.
Back in 2000-ish, a summer holiday job I was doing for an hourly pay of… I can't remember exactly, but perhaps the equivalent of £10k/year for the full-timers… one of my coworkers said he'd bought a plasma TV for his 1- or 2-year old son. Those things were considered expensive luxuries back then.
Man, the Metaverse... what a fad. I almost can't even believe that WAS a thing. We had Habbo, and then we had Second Life, and NOW!!!!!!.... * deafening silence *
VR is a gadget, the same way that 4D cinemas are gadget. Ok it's +/- fun the first time to receive water droplets in your face while watching a movie, or for the cinema to spread fake fart smell, but this is not the experience you want to have every day.
Apparently only a single-digit % of people who purchased AR headsets are using them once in a month (source: Valve developer Chet Faliszek)
A (dystopian) future is more likely the Neuralink injecting visual signals or thoughts. Even an ultra-light HMD gives the Glasshole feeling, besides not being comfortable to wear.
> but this is not the experience you want to have every day.
It’s been the experience I’ve been having every day since the original oculus devkit. I’ve rarely touched a flatscreen title since. Eventually I hope they’ll become good enough to work in when I travel.
edit
In saying that, i've always maintained its not ready for normies yet, and that we were probably a good decade or two away from the point where its going to be in a format/design that the mass market would accept/have interest in. In fact in many cases, I still actively reccomend against purchase because I know it's still kinda incuating along.
There are 12 players online now, 22 players "peak"...
A lot of people have tried VR, and the consensus is generally that VR is fun to try once but then:
Is it really worth spending 1000 USD (headset only + whatever gaming PC you need with it) <-> 4000 USD (vision pro) for something you'd use only few times per year ?
To follow the hardware news related to something you don't plan to purchase nor use doesn't really make sense.
>Is it really worth spending 1000 USD (headset only + whatever gaming PC you need with it) <-> 4000 USD (vision pro) for something you'd use only few times per year ?
The Quest 2 is a perfectly capable VR device and is only $250.
>To follow the hardware news related to something you don't plan to purchase nor use doesn't really make sense.
You are on hackernews.. a site that has talked about the Quest 3 A TON.
World GDP/Capita is $12,234. Why would you use world population instead of US population? This discussion is clearly not serious so I'm not gonna respond further
The promise of the former is a unified platform where e.g. a virtual magic sword bought in an RPG also works in an FPS made by an unrelated developer, and a virtual gun bought in the later also works in the former, and this is also your work collaboration environment. And now I'm thinking of ABK's boss fight sketch: https://youtu.be/w6u_EJa_sZE?si=wlYD8EhRd_PLm39l
HN title seems editorialized as compared to the article title ("Meta’s new goal is to build artificial general intelligence" or the article headline is similar but with Mark Zuckerberg as the subject.)
The HN submitted title ("Meta will have a stockpile of almost 600k GPUs by the end of 2024") is one specific sentence in the article.
We know OpenAI and Azure was struggling to get enough GPU. That was implied not just by their words but also by action. And considering these two companies are most aggressive and making most money out of this AI. If GPUs are available they would have been able to buy it.
Build their own? It's what Microsoft, Google, and AWS are doing.
>Nvidia's sales are only limited by the number of wafers they can get from TSMC.
No, they're limited by the cost per operation vs. Facebook building their own. The cloud providers have already decided it's cheaper to do it themselves. Sure they'll keep buying GPUs for general public consumption but that may eventually end too.
There certainly is a lot of value in the CUDA ecosystem, today. The problem is that when all the big companies are buying up hundreds of thousands of GPUs, that doesn't leave much for anyone else.
Sane business people will look to decentralize their compute over time and not be reliant on a single provider. AMD will be able to take advantage of that and they've already stated that is their focus going forward.
ROCm/HIP are getting incrementally better, MI300x have 192GB and benchmarks are looking good, the only problem is that nobody has access to the higher end hardware from AMD today. That's why I'll have MI300x, for rent, soon.
That's a big issue in AMD land imho. Everyone can pickup a 200$ GPU (talking about the RTX 3050) which will behave like a scaled down A100 and get started playing around with CUDA. You can't really do that with AMD GPUs, their cheapest officially supported GPU is the 7900 XTX and that has a different architecture than the data center ones.
Nvidia has a vested interest in FB being beholden to their chips, so much so that it's worth giving them a discount to ensure it happens, and human nature being human nature a face saving discount has to be offered.
Oh man if Elon had billions in H100s we might actually see that happen. And I’m no fan of Elon but I’m also no fan of cryptocurrency these days. Might be worth it just to watch the crypto world burn.
A successful 51% attack on a major cryptocurrency would not necessarily be that impactful. So what if Elon can doublespend? He would need a lot of crypto, a counterparty, and the strong desire to waste money. Large miners could already collude to do it, it just is not in their interest.
Shorting Bitcoin has been possible for a decade now. The capex to pull off the would be in the billions though, and the value of that investment is tied directly to the price of Bitcoin.
That said, the attack you describe has happened for much smaller cryptos. I'm not saying it can't happen, I'm saying there's no reason to assume it would be a huge threat to Bitcoin, because the actual risk for a user is vanishingly small. There are much bigger threats to Bitcoin's valuation that are far more plausible, such as government crackdowns.
The question, and this thread, was entirely about if it this is enough compute to do it. Not if it was, in your opinion, a threat worth worrying about.
Depends on how much compute there is to mine it. Not that many valuable cryptos still use GPU PoW. You also need a counterparty to actually profit from it.
I wonder how many of those chips were acquired to run metaverse stuff. Should be lots of overlap between rendering graphics and running cuda based models.
I'm interested in seeing how the behemoths that are Meta and Google catch openai. I think it's a question of when, not if. Both companies just have a ridiculous amount of resources to throw behind these efforts. At least meta is releasing their stuff as "open source". We'll see how they justify putting out these models for free, or if it's purely about undercutting openai.
Almost no overlap: this metaverse thing just needs classical CPU servers (and not a lot of them considering the minimal user activity there).
For now Google is still late to the party (full proprietary, and nobody has seen the supposedly good model called Ultra, only an average one called Pro), and Meta is actually the company that has pushed the field forward for all companies (with LLaMA).
I’m more bullish on Meta’s AI efforts than OpenAI’s at this point. Everything open source can flow back into what they’re doing, whereas OpenAI seems focused on staying locked down, while diluting their core product in myriad ways.
OpenAI got all of the positive publicity for its social mission yet in short time we clearly see that Meta has done more for democratizing access to deep learning and will continue to lead on this front. The cost of open sourcing models is far more than just development cost. They're spending millions of dollars training models with that fleet of H100s. This makes open source AI much more costly, and generous.
As Meta has always done. Their contributions to open source ML have always been above and beyond everyone else and they have one of the absolute best teams in the industry.
From a naive perspective, it seems that true research/advances in AI (methods of training, etc) aren't necessarily related to model size. It seems that the goal of "building a big model that everyone else converges to because the training data is the same" doesn't have all that much value, especially since you could wait a couple years, do it all for a fraction of the price, and catch up immediately. Meta doesn't have an AI product yet, so it's not like they would be loosing money.
I suspect this is more about talent attraction/retention.
CPUs are rapidly depreciating, hard drives are rapidly depreciating, SSDs are doubly rapidly depreciating, with this logic no hardware buy would make sense.
Computer hardware has always been rapidly depreciating. You'd always get much more (performance/capability) for the same money just a few months to 1-2 years down the road. GPUs have been a complete outlier in this area for around ~8 years, and even they depreciate relatively quickly still.
I am going to be buying these by the truckload on ebay three years from now.
The current generation of GPUs is definitely going to have a long usable lifetime. Manufacturers finally have HBM totally figured out and yielding/aging well. Today's GPUs are the analog of the 28nm logic node -- it was the sweet spot for an absurdly long time.
There will be something faster/lower-power 2-3 years from now that will cause the BigCos to cycle their fleet, but it won't be anywhere near the VRAM/MemBandwidth boost we have today compared to 2-3 years ago. Their data centers are power-constrained, so they'll upgrade regardless, and the rest of us will feast on the windfall.
Hopefully Llama 3 and Llama 4 open models will be released soon.
For all of Meta’s faults, release powerful LLMs that users can run and modify on their own systems is a huge benefit to keeping AI from being entirely locked away and heavily censored by big corporations.
...rendering every competitor's massive investments worthless. Zuckerberg's thinking must be that Meta's competitors are more susceptible to disruption by broadly accessible AGI, and that if everyone has access to state-of-the-art AGI, then no one will be able to gain a new kind of competitive advantage from it.
And what are those investments worth? Quests have great quality with nearly no-profit price and who uses it. Their Horizon platform that even employees hate.
Everyone is building something, also Tesla is building FSD, I am not sure why journalists decide to give up their profession’s dignity and become CEOs extended PR department
Article is a bit of a nothingburger, here's the quote in question from Zuck's Instagram:
"Some updates on our AI efforts. Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit. We're bringing our two major AI research efforts (FAIR and GenAI) closer together to support this. We're currently training our next-gen model Llama 3, and we're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year -- and overall almost 600k H100s equivalents of compute if you include other GPUs."
I feel like this is a subtle attempt to move the goalposts on what is meant by AGI. Regardless of whether the final product is truly an AGI (and I'm guessing it won't be) my guess is that it will be branded as such.
Say what you want about Facebook, the size of their dataset and computational resources definitely make them competitive, and their data science and ML teams have been always top notch. I think the Verge is missing the mark with the headline and general focus of the article. "Building AGI" is whatever, like half the companies with enough GPUs are claiming that and AGI is like more poorly-defined than "metaverse". The more interesting point seems to be this general incoherence with building chatbots and trying to run a social media company.
>Meta is still a metaverse company. It’s the biggest social media company in the world. It’s now trying to build AGI. Zuckerberg frames all this around the overarching mission of “building the future of connection.”
This is such "Verge" writing. I'm by no means bearish on VR, but that whole passage is so unreflective and uncritical it's almost a satire of journalistic fluff. Chatbots that fill social media with greater and greater amounts of garbage content is just a nightmare. Bot content is already one of the reasons people are retreating into groupchats. The blurring of AI and human interaction leads to accountability problems. Hell, Snapchat and Discord basically already tried this to enormous backlash. The fact that this is entirely antagonistic with "building the future of connection" goes essentially unacknowledged.
There is something interesting with the fact that Facebook is more open to open-source, this is fairly credible actually given the quality and quantity of the company's open-source contributions. But I genuinely think LLMs are most useful as an applied technology, and the applications listed here are frankly uninspiring.
Saying they make it “open source” in the same article where they say they need “350k high end GPUs to build it”. Is the equivalent of saying: ”we offer free nuclear submarine driving lessons”.
I know you don’t need as many resources for inference as for training. But still…
What do you mean “but still…”? It’s a pretty important distinction. Meta does indeed use their massive GPU farms to train models and then release the weights for free and people indeed run inference on prosumer hardware
How different is saying they built React using 100-500+ developer years of effort and then open sourced it. What they are releasing is what is needed by most of the people looking for open models.
That's such a crazy scale to consider that it makes me wonder how easy it is to maintain control over that many cards, and how easy it would be for someone at a company of this scale to become a purchasing tunnel to countries with sanctions.
They'd notice a couple thousand of these if they were to vanish or never arrive. Anything less is fairly meaningless for major economies.
China for example needs epic scale numbers of GPUs to power its economy going forward. The equivalent of many millions of H100s for a $20 trillion economy looking to advance rapidly.
Given the restricted production globally (Nvidia production bottleneck, with only a few places on earth that can produce something like this), until China can produce their own very high-end GPUs their economy is going to be held back by the lack of capacity. Tens of thousands of high-end GPUs slipping through isn't going to cut it, that simply doesn't matter very much.
You can prevent China from getting a million H100s. You can't prevent them from getting ten thousand of them from many different sources over time.
I imagine you don't expect a serious answer to that anyway, but to be clear: anyone talking about AI timelines seriously would not be expressing so much certainty because it's not possible to know AI timelines with certainty right now.
> why there's so much insistence from business that magic happens at scale with LLMs
It's already happening. See latest Google layoffs. They are automating a lot of things. Most people don't realize it, but the change is going to be dramatic.
> We're a long way from AGI
This is a big question, what is AGI? LLMs are quite generic and 'intelligent'. Not human-like, but. Next is going to be incremental evolution. Till we find other, non-verbal, ways of 'thinking' and put them together. That's going to be a breakthrough. Interesting, terminator-like embodiment isn't a requirement for AGI, nor is stable 'personality'.
> It's about using your environment to gain an evolutionary advantage.
I don't think that's right either (it sounds like a description of adaptation), and I don't think your description of LLMs is fair, even though I'm fairly sure they're not AGI and won't scale to AGI.
Intelligence is more like the ability to generalize skills, applying knowledge gained in one scenario to another scenario.
> Intelligence is more like the ability to generalize skills, applying knowledge gained in one scenario to another scenario.
Hmm.. you are talking about LLMs.. They are the most generic thing we have right now (Jan 2024) LLMs have limitations, like learning on the fly isn't their strong side. But the same with brain, it consists of limited components, that's only together they work well. LLMs can be a part of the solution, if we can't find something better.
> using your environment to gain an evolutionary advantage.
That's more like robotics. Except for evolution part. Does AGI require breeding? Software can easily multiply itself. That's hardware is the problem then.
Think about what incentives do you have to live - it may sound rough but pain and ultimately death are the ultimate things everyone is trying to avoid.
Intelligence emerges as you are trying to survive longer. Reproduction is the ultimate way of cheating death.
Environment does not necessarily needs to mean physical environment, but until the "AI" does not recognize that it is in danger of nonexisting and starts to behave in ways to avoid it it cannot, imho, make the leap to AGI and it's just a really sophisticated tool.
> Intelligence emerges as you are trying to survive longer.
That's not a given - I don't even know if it's true. The longest-lived species are not very intelligent, relative to humans. Intelligence is a tool that may or may not evolve in organic species. Frankly, that has very little to do with defining what artificial intelligence is.
361 comments
[ 2.6 ms ] story [ 293 ms ] threadSo...what happened to that whole "Metaverse" thing? Is it time to rename the company again?
AI will be used to enhance that e.g. generated avatars, autonomous agents, hand/body tracking etc.
Strong adoption by the younger TikTok generation which is an area that Meta has been desperate to bring back into their fold.
Also the product is pretty impressive. The camera quality is really good and the AI features genuinely useful. Definitely caused many in the industry to wonder if that form factor could be the future of the AR industry in the short term as well as the long term.
Less than 10% retention after two years.
I may be repeating my own errors with the following as I said much the same for the iPhone and the Apple Watch when they were new, but…
The prices Apple are asking for seem excessive given what the products actually do, surely the cheaper alternatives are going to be what really matters?
(In this case, cheaper alternatives would include the Meta headsets).
Watched a guy spend $6K on iPads for his kids like it was a stocking stuffer. Took 15 minutes as he had to call the wife on if 1TB was enough. Felt bad I was just buying an Air so wandered to let the sales kid work him.
Kid said it was an overage day.
You've just reminded me how weird I am with money.
Back in 2000-ish, a summer holiday job I was doing for an hourly pay of… I can't remember exactly, but perhaps the equivalent of £10k/year for the full-timers… one of my coworkers said he'd bought a plasma TV for his 1- or 2-year old son. Those things were considered expensive luxuries back then.
I guess people like him are the norm.
Ah, yes, the inevitability of Apple success.
Just as everyone switched from cell phones to PDAs after 1993.
Generally, they get it right a lot of the time. More than others!
But argumentum ad Apple to convince of the inevitability of technological shifts is insane.
Apparently only a single-digit % of people who purchased AR headsets are using them once in a month (source: Valve developer Chet Faliszek)
Which have you had experience with? And, do you believe whatever you wore was as good as it gets?
It’s been the experience I’ve been having every day since the original oculus devkit. I’ve rarely touched a flatscreen title since. Eventually I hope they’ll become good enough to work in when I travel.
edit
In saying that, i've always maintained its not ready for normies yet, and that we were probably a good decade or two away from the point where its going to be in a format/design that the mass market would accept/have interest in. In fact in many cases, I still actively reccomend against purchase because I know it's still kinda incuating along.
If nobody can hear the sound of a tree falling, does it still make a sound ?
Look for example Among US VR (an excellent game):
https://steamdb.info/app/1849900/charts/
There are 12 players online now, 22 players "peak"...
A lot of people have tried VR, and the consensus is generally that VR is fun to try once but then:
Is it really worth spending 1000 USD (headset only + whatever gaming PC you need with it) <-> 4000 USD (vision pro) for something you'd use only few times per year ?
To follow the hardware news related to something you don't plan to purchase nor use doesn't really make sense.
The Quest 2 is a perfectly capable VR device and is only $250.
>To follow the hardware news related to something you don't plan to purchase nor use doesn't really make sense.
You are on hackernews.. a site that has talked about the Quest 3 A TON.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
Looking at VR Chat though, it's posting record highs in user count, hitting 52,956 over new years, with steady year over year growth for 5 years now.
https://steamdb.info/app/438100/charts/#all
The promise of the former is a unified platform where e.g. a virtual magic sword bought in an RPG also works in an FPS made by an unrelated developer, and a virtual gun bought in the later also works in the former, and this is also your work collaboration environment. And now I'm thinking of ABK's boss fight sketch: https://youtu.be/w6u_EJa_sZE?si=wlYD8EhRd_PLm39l
I'm gonna get downvoted for this comment because this is not Reddit, but I had to say it.
The HN submitted title ("Meta will have a stockpile of almost 600k GPUs by the end of 2024") is one specific sentence in the article.
That's approx $15b worth of H100 GPUs.
Presumably all the chip supply issues regarding autos have been resolved, and yet prices have risen 30% in a decade, and there’s no reversal.
What are they going to do? Buy AMD, yeah right.
Nvidia's sales are only limited by the number of wafers they can get from TSMC.
Build their own? It's what Microsoft, Google, and AWS are doing.
>Nvidia's sales are only limited by the number of wafers they can get from TSMC.
No, they're limited by the cost per operation vs. Facebook building their own. The cloud providers have already decided it's cheaper to do it themselves. Sure they'll keep buying GPUs for general public consumption but that may eventually end too.
There is no value into the NVIDIA-part by itself, only the raw power is interesting.
If tomorrow this is AMD, or China-Town chip, it's perfectly fine.
I wouldn't miss the CUDA toolkit mess.
We are not, because there's clearly value in the CUDA ecosystem.
Sane business people will look to decentralize their compute over time and not be reliant on a single provider. AMD will be able to take advantage of that and they've already stated that is their focus going forward.
ROCm/HIP are getting incrementally better, MI300x have 192GB and benchmarks are looking good, the only problem is that nobody has access to the higher end hardware from AMD today. That's why I'll have MI300x, for rent, soon.
Some people will pay more to be first in line.
Given that GPU production is mostly sold out, and that giving META a bigger discount would simply mean losing money from other purchasers.
That said, the attack you describe has happened for much smaller cryptos. I'm not saying it can't happen, I'm saying there's no reason to assume it would be a huge threat to Bitcoin, because the actual risk for a user is vanishingly small. There are much bigger threats to Bitcoin's valuation that are far more plausible, such as government crackdowns.
I'm interested in seeing how the behemoths that are Meta and Google catch openai. I think it's a question of when, not if. Both companies just have a ridiculous amount of resources to throw behind these efforts. At least meta is releasing their stuff as "open source". We'll see how they justify putting out these models for free, or if it's purely about undercutting openai.
For now Google is still late to the party (full proprietary, and nobody has seen the supposedly good model called Ultra, only an average one called Pro), and Meta is actually the company that has pushed the field forward for all companies (with LLaMA).
This is the first time I'm hearing this, unless you mean the fact that it leaked to the public. How was LLaMa pushing the field forward otherwise?
The idea is that the metaverse will be filled with AI avatars.
Good (as written), or God (at first glance)?
Microsoft name change incoming?
https://www.tomshardware.com/news/nvidia-h100-benchmarkedin-...
I suspect this is more about talent attraction/retention.
The current generation of GPUs is definitely going to have a long usable lifetime. Manufacturers finally have HBM totally figured out and yielding/aging well. Today's GPUs are the analog of the 28nm logic node -- it was the sweet spot for an absurdly long time.
There will be something faster/lower-power 2-3 years from now that will cause the BigCos to cycle their fleet, but it won't be anywhere near the VRAM/MemBandwidth boost we have today compared to 2-3 years ago. Their data centers are power-constrained, so they'll upgrade regardless, and the rest of us will feast on the windfall.
Don't ninja-bid on my auctions.
For all of Meta’s faults, release powerful LLMs that users can run and modify on their own systems is a huge benefit to keeping AI from being entirely locked away and heavily censored by big corporations.
Im building a planet
Are they also working on world peace?
"Some updates on our AI efforts. Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit. We're bringing our two major AI research efforts (FAIR and GenAI) closer together to support this. We're currently training our next-gen model Llama 3, and we're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year -- and overall almost 600k H100s equivalents of compute if you include other GPUs."
>Meta is still a metaverse company. It’s the biggest social media company in the world. It’s now trying to build AGI. Zuckerberg frames all this around the overarching mission of “building the future of connection.”
This is such "Verge" writing. I'm by no means bearish on VR, but that whole passage is so unreflective and uncritical it's almost a satire of journalistic fluff. Chatbots that fill social media with greater and greater amounts of garbage content is just a nightmare. Bot content is already one of the reasons people are retreating into groupchats. The blurring of AI and human interaction leads to accountability problems. Hell, Snapchat and Discord basically already tried this to enormous backlash. The fact that this is entirely antagonistic with "building the future of connection" goes essentially unacknowledged.
There is something interesting with the fact that Facebook is more open to open-source, this is fairly credible actually given the quality and quantity of the company's open-source contributions. But I genuinely think LLMs are most useful as an applied technology, and the applications listed here are frankly uninspiring.
I know you don’t need as many resources for inference as for training. But still…
China for example needs epic scale numbers of GPUs to power its economy going forward. The equivalent of many millions of H100s for a $20 trillion economy looking to advance rapidly.
Given the restricted production globally (Nvidia production bottleneck, with only a few places on earth that can produce something like this), until China can produce their own very high-end GPUs their economy is going to be held back by the lack of capacity. Tens of thousands of high-end GPUs slipping through isn't going to cut it, that simply doesn't matter very much.
You can prevent China from getting a million H100s. You can't prevent them from getting ten thousand of them from many different sources over time.
The lack of meaningful details in these announcements makes me pessimistic.
It's already happening. See latest Google layoffs. They are automating a lot of things. Most people don't realize it, but the change is going to be dramatic.
> We're a long way from AGI
This is a big question, what is AGI? LLMs are quite generic and 'intelligent'. Not human-like, but. Next is going to be incremental evolution. Till we find other, non-verbal, ways of 'thinking' and put them together. That's going to be a breakthrough. Interesting, terminator-like embodiment isn't a requirement for AGI, nor is stable 'personality'.
Intelligence is not about parroting an answer you've seen before. It's about using your environment to gain an evolutionary advantage.
I don't think that's right either (it sounds like a description of adaptation), and I don't think your description of LLMs is fair, even though I'm fairly sure they're not AGI and won't scale to AGI.
Intelligence is more like the ability to generalize skills, applying knowledge gained in one scenario to another scenario.
the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (such as tests)
Manipulating the environment afaik is to increase your chances of survival.
Hmm.. you are talking about LLMs.. They are the most generic thing we have right now (Jan 2024) LLMs have limitations, like learning on the fly isn't their strong side. But the same with brain, it consists of limited components, that's only together they work well. LLMs can be a part of the solution, if we can't find something better.
That's more like robotics. Except for evolution part. Does AGI require breeding? Software can easily multiply itself. That's hardware is the problem then.
Intelligence emerges as you are trying to survive longer. Reproduction is the ultimate way of cheating death.
Environment does not necessarily needs to mean physical environment, but until the "AI" does not recognize that it is in danger of nonexisting and starts to behave in ways to avoid it it cannot, imho, make the leap to AGI and it's just a really sophisticated tool.
That's not a given - I don't even know if it's true. The longest-lived species are not very intelligent, relative to humans. Intelligence is a tool that may or may not evolve in organic species. Frankly, that has very little to do with defining what artificial intelligence is.
What exactly did they automate?
Their expenses haha
Some more pressing questions are:
* What new capabilities emerge as the models get better and better at predicting (i.e. loss goes down)?
* How much will it cost to train increasingly large models? And to run inference on them?
* How difficult will it be to find or generate more and more high quality data?
what are they then? I thought everyone was firmly in the "let's train with more data and see what happens" camp
https://arxiv.org/abs/2001.08361
How loss translates into higher-level capabilities is anyone's guess.