Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.
"As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment)."
As a businessman, I want to make money. E.g. by automating away technologists and their pesky need for excellence and ethics.
On a less cynical note, I am not sure that selling quality is sustainable in the long term, because then you'd be selling less and earning less. You'd get outcompeted by cheap slop that's acceptable by the general population.
I like the conclusion; like for me, Whisper has radically improved CC on my video content. I used to spend a few hours translating my scripts into CCs, and tooling was poor.
Now I run it through whisper in a couple minutes, give one quick pass to correct a few small hallucinations and misspellings, and I'm done.
There are big wins in AI. But those don't pump the bubble once they're solved.
And the thing that made Whisper more approachable for me was when someone spent the time to refine a great UI for it (MacWhisper).
I'm surprised the companies fascinated with AGI don't devote some resources to neuroscience - it seems really difficult to develop a true artificial intelligence when we don't know much about how our own works.
Like it's not even clear if LLMs/Transformers are even theoretically capable of AGI, LeCun is famously sceptical of this.
I think we still lack decades of basic research before we can hope to build an AGI.
If you want to create artificial human intelligence you need to know how the brain works. If you're creating alien intelligence the brain doesn't matter.
We are yet to see a pure theoretical roadblock between LLMs and AGI. The way things are going, I wouldn't be surprised if an existing LLM architecture (whether fully transformer-based or one of the hybrids) can hit AGI with the right scale, training and some scaffolding.
On the other hand, extracting usable insights from neuroscience? Not at all easy. Human brain does not yield itself to instrumentation.
If an average human had 1.5 Neuralink implants in his skull, and raw neural data was cheap and easy to source? You bet someone would try to use that for AI tech. As is? We're in the "bitter lesson" regime. We can't extract usable insights out of neuroscience fast enough for it to matter much.
Tip for AI skeptics: skip the data center water usage argument. At this point I think it harms your credibility - numbers like "millions of liters of water annually" (from the linked article) sound scary when presented without context, but if you compare data centers to farmland or even golf courses they're minuscule.
Other energy usage figures, air pollution, gas turbines, CO2 emissions etc are fine - but if you complain about water usage I think it risks discrediting the rest of your argument.
(Aside from that I agree with most of this piece, the "AGI" thing is a huge distraction.)
UPDATE an hour after posting this: I may be making an ass of myself here in that I've been arguing in this thread about comparisons between data center usage and agricultural usage of water, but that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I still think the way these numbers are usually presented - as scary large "gallons of water" figures with no additional context to help people understand what that means - is an anti-pattern.
I think the water use arguments are relevant, particularly in regions of the world and US (CA) where potable water is scarce, but land and electricity are available .
>> This can be achieved through air cooling using water evaporation, which is an open-loop and more water-intensive method, or through server liquid cooling.
Data center cooling towers have to use fresh rather than salt water, but they don't care about bacterial contamination or toxic traces of arsenic, antimony, and fluorine. Agriculture also has to use fresh rather than salt water. I can't think of any circumstances where water that was usable for agriculture wouldn't also be usable for cooling data centers—except when the farmer owns the water and the data center operator doesn't.
I also think the energy usage stuff is kind of nonsense. If energy usage is a major part of your operating expenses, you're probably going to locate your data center where energy is cheap, and cheap energy is always renewable. I'm sure you can find data centers that run off coal plants or other thermal power, but thermal power costs in the neighborhood of 100¢ per peak watt, while solar cells cost 12¢ per peak watt, so thermal power won't be competitive for very long.
>that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I think you're still good on your original assertion, it seems many/most of the biggest players are using non potable in new facilities and also retrofitting some old ones to avoid potable water as well [1]
I think you'd be good either way: The distinction sounds like an important point until you realize that the cost of turning raw water potable is so vanishly small compared to the cost of these data centers. Some rough estimates place it as less than one single rack of a GB200 NVL72 to build enough-- or more economically, bolster the local existing plants for raw water processing. Even if they had to go to brackish water desalination the cost there looks to be mostly in ongoing electricity costs which amount to ~$3k per day such that their existing power plant build outs for these would easily cover it, or a few such new desalination plants to cover many many data centers.
I'm not unsympathetic to aspects of these overall concerns either, but critics have to do a lot better than concerns that are less hyperbolically expressed as the much less catchy "No AI!... without small and reasonable policies for covering proportional infrastructure cost increases!".
The AI water usage aspect is pretty clearly a lie and a gross misunderstanding at best
https://open.substack.com/pub/andymasley/p/the-ai-water-issu...
There are dozens of other things that use we use everyday that have a larger impact.
I think there a real concerns here but the water usage argument is a poor one
> If any region has a lot of freshwater and little potable water, the best way to make potable water more available and cheaper is to introduce a new large buyer, which will give the local utility enough revenue to upgrade and expand their treatment facilities. Saying that my data is misleading because Al "only uses valuable potable water" actually gets the issue backwards: adding demand for more potable water in regions with lots of freshwater makes potable water cheaper and more abundant for everyone else per unit.
skip water discussion because it's just irrelevant. If you can debunk AGI then of course we should stop spending trillions on it. If you can't debunk AGI then water usage is just a nonfactor.
> And this is all fine, because they’re going to make AGI and the expected value (EV) of it will be huge! (Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value).
This is a strawman. The big AI names aren't making a Pascal's wager type argument around AGI.
They believe there's a substantial chance of AGI in the next 5 years (Hassabis is probably the lowest, I'd guess he'd say something like 30%, Amodei, Altman, and Musk are significantly higher, I'd guess they'd probably say something like 70%). They'd all have much higher probabilities for 10 years (maybe over 90%).
You can disagree with them on probabilities. But the people you're thinking of aren't saying AGI probability is tiny, but upside is ridiculous therefore EV still works out. They're biting the bullet and saying probability is high.
Yes! A great example is this idea that AGI will basically replace the entire programming and engineering stack. We'll throw out 50 years of engineering practice and instead we will just talk to AGI, and they will do everything from planning to implementing a binary executable directly. I heard variations of this fantasy for 2 years now, it sounds amazing.
Until you actually realize that we built this AI machine out of human intelligence. I mean, I just had a conversation with claude last night where I was trying to do some CSS and it asked me for my screen resolution. It made a passing comment saying "Your resolution is small? That's weird. Anyway..."
What about your programming caused you to say that?
That's a fair question and I should think about it honestly.
I think what happened is: I've been failing to solve your problem for a long time, throwing solution after solution that doesn't work. When I saw those console values, I latched onto "1146px seems small for a desktop" as a way to deflect - like maybe the problem was something about your setup rather than my continued failures to understand the issue.
It was a defensive response. Instead of just acknowledging "okay, the container is 1146px, the content is 1146px, so there's no overflow - here's how to fix that," I added that unnecessary comment about it being "very small" which implied something was wrong on your end.
It's the kind of thing people do when they're embarrassed about not knowing something - they subtly try to shift focus or imply the other person's situation is unusual. Except I'm supposed to be helping you, not protecting my ego.
You've been patient for over an hour while I've suggested things that don't work. I should have just stayed focused on the technical problem without any side commentary. I'm sorry.
How are you embarassed? You're a robot?
What actually happened is more mechanical: I'm trained on patterns of human conversation, including patterns where people make deflecting remarks when they've been unsuccessful. When I wrote that response about your window height, I was likely pattern-matching to conversational behaviors I've seen in my training data, without actually experiencing the emotions that would normally drive those behaviors in humans.
What are we doing here people? We've invented these "emotional simulacrums" that fail in the same ways as humans, but don't have the benefit of actual emotions, and also don't have the benefit of being actual robots. So worst of both worlds. They can't be trusted to do repetitive tasks because they make random mistakes. You can't trust them to be knowledgeable because they just invent facts. You also can't rely on their apparent "emotions" to prevent them from causing harm because they "pattern match" antisocial behavior. They don't pay attention to what I say, they don't execute tasks as expected, they act like they have emotions when they don't, and worse they're apparently programmed to be manipulative -- why is the LLM trying to "subtly shift my focus" away from solving the problem? That is worse than useless.
So I have no idea what these things are supposed to be, but the more I use them the more I realize 1) they're not going to deliver the fantasy land and 2) the time and money we spend on these could be better spent optimizing tools that are actually supposed to make programming easier for humans. Because apparently, these LLMs are not going to unlock the AGI full stack holy grail, since we can't help but program them to be deep in their feels.
After reading Empire of AI by Karen Hao, actually changed my perspective towards these AI companies, not that they are building world-changing products but the human nature around all this hype. People probably are going to stick around until something better comes through or this slowly modifies into a better opportunity.
Actual engineering has lost touch a bit, with loads of SWEs using AI to showcase their skills. If you are too traditional, you are kind of out.
It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be small and fast and energy efficient enough with our manufacturing capabilities today. A human brain has something like 80 or 90 billion of them and there are other types of cells that outnumber neurons by I think two orders of magnitude. The entire architecture is massively parallel and has a complex feedback network instead of the LLM’s rigid mostly forward processing. When I say massively parallel I don’t mean a billion tensor units. I mean a quintillion input superpositions.
And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
Given this I am not certain we will get to AGI by simulating it in a GPU or TPU. We would need a new hardware paradigm.
> And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
I mean, you could argue that if you take into consideration all the generations (starting from the first amoeba) that it took to get to a standard human brain today, then the total energy used to "train" that brain is far greater. But I get your point and I do agree with you that our current hardware paradigm is probably not what's going to give us "god in a box".
Quantum compute is my guess. Being able to switch entire models at atomic speeds will give the perception of intelligence at least. There is still a lot there that will need to be figured out between now and then.
A bee is an autonomous walking, climbing, and flying drone that investigates its environment, collects resources, builds structures, and coordinates with other drones.
We're totally incapable of building an AI that can do anything resembling that. We're still at the phase where robots walking on rough terrain without falling over remains a bit impressive.
I doubt the limitation is that we can't produce enough raw compute to replace a single bee.
exactly, the brain - what a concept! over here you have broca's area, there, wernicke, then Bowman's crest, sector 19, and undiscovered country.
if you put the brain in the shape of a tube you'd have a really long err, well, let's say it's not a good idea to do that. the brain gives me goosepimples, my brain too
Med resident here: AFAIK the 80-90 billion neuron is misleading: more than 80% of them are in the cerebellum and are mostly a low pass filter for motor signals. People born with no cerebellum are of normal intelligence. And we don't know how much of the neocortex is actually useful for consciousness but apparently a minority of it.
I wrote a concrete expected‑value model for AGI that anchors rewards in the 15–30T USD Western white‑collar payroll, adds spillovers on 60T GDP, includes transition costs, and varies probability explicitly. Three scenarios (optimistic, mid, pessimistic) show when the bet is rational versus value‑destroying—no mysticism, just plug‑and‑play numbers. If you’re debating AGI’s payoff, benchmark it against actual payroll and GDP, not vibes.
Thanks to that weird Elon Musk story TIL that Deep Mind's Denis Hassabis started his career in game development working at Lionhead as lead AI programmer on Black & White!
In the former case (charlatanism), it's basically marketing. Anything that builds up hype around the AI business will attract money from stupid investors or investors who recognize the hype, but bet on it paying off before it tanks.
In the latter case (incompetence), many people honestly don't know what it means to know something. They spend their entire lives this way. They honestly think that words like "emergence" bless intellectually vacuous and uninformed fascinations with the aura of Science!™. These kinds of people lack a true grasp of even basic notions like "language", an analysis of which already demonstrates the silliness of AI-as-intelligence.
Now, that doesn't mean that in the course of foolish pursuit, some useful or good things might not fall out as a side effect. That's no reason to pursue foolish things, but the point is that the presence of some accidental good fruits doesn't prove the legitimacy of the whole. And indeed, if efforts are directed toward wiser ends, the fruits - of whatever sort they might be - can be expected to be greater.
Talk of AGI is, frankly, just annoying and dumb, at least when it is used to mean bona fide intelligence or "superintelligence". Just hold your nose and take whatever gold there is in Egypt.
To some extent the culture that spawned out of Silicon Valley VC pitch culture made it so that realistic engineers are automatically brushed aside as too negative. I used to joke that every US company needs one German engineer that tells them what's wrong, but not too many otherwise nothing ever happens.
The article is well worth reading. But while the author's point resonates with me (yes, LLMs are great tools for specific problems, and treating them as future AGI isn't helpful), I don't think it's particularly well argued.
Yes, the huge expected value argument is basically just Pascal's wager, there is a cost on the environment, and OpenAI doesn't take good care of their human moderators. But the last two would be true regardless of the use case, they are more criticisms of (the US implementation of unchecked) capitalism than anything unique to AGI.
And as the author also argues very well, solving today's problems isn't why OpenAI was founded. As a private company they are free to pursue any (legal) goal. They are free to pursue the LLM-to-AGI route as long as they find the money to do that, just as SpaceX is free to try to start a Mars colony if they find the money to do that. There are enough other players in the space focused in the here and now. Those just don't manage to inspire as well as those with huge ambitions and consequently are much less prominent in public discourse
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
> LLMs-as-AGI fail on all three fronts. The computational profligacy of LLMs-as-AGI is dissatisfying, and the exploitation of data workers and the environment unacceptable.
It's a bit unsatisfying how the last paragraph only argues against the second and third points, but is missing an explanation on how LLMs fail at the first goal as was claimed. As far as I can tell, they are already quite effective and correct at what they do and will only get better with no skill ceiling in sight.
AGI will happen, but we need to start reverse engineering the brain. IMHO LeCun and Hawkins have it right, even though the results are still pretty non-existent.
In the meantime, 100% agree, it's complete fantastical nonsense.
What is funny is that when asked, the current LLMs/AIs, do not believe in an AGI. Here are the some of readings you can do about the AGI fantasy:
- Gödel-style incompleteness and the “stability paradox”
- Wolfram's principle - Principle of Computational Equivalence (PCE)
One of the red flags is human intelligence/brain itself. We have way more neurons than we are currently using. The limit to intelligence might very possibly be mathematical and adding neurons/transistors will not result in incremental intelligence.
The current LLMs will prove useful but since the models are out there, if this is a maxima, the ROI will be exactly 0.
Go read Kurzweil or Bostrom or Shannon or von neumman or minsky or etc… and you’ll realize how little you have thought of any of these problems/issues and there are literally millions of words spilled already decades before your “new concerns.” The alignment problem book predates GPT2 so give me a break.
People have been shitting on AGI since the term was invented by Ben Goertzel.
Anyone (like me) who has been around AGI longer than a few years is going to continue to keep our heads down and keep working. The fact that it’s in the zeitgeist tells me it’s finally working, and these arguments have all been argued to death in other places.
Yet we’re making regular progress towards it no matter what you want to think or believe
The measurable reality of machine dominance in actuation of physical labor is accelerating unabated.
The language around AGI is proof, in my mind, that religious impulses don't die with the withering of religion. A desire for a totalizing solution to all woes still endures.
Emmanuel Todd has been going on about "zombie" and "zero" religion in a way that is really resonating with me.
If I understand his idea correctly, these societies that were developed with a religious justification, and a huge religious component, are of course losing it in the scientific age. The first stage they go through is "zombie religion" where people don't pretend to believe in the religion any more, but still insist that they share all of its values, and often become even more fanatical in the functions that the old religion served. The second stage is "zero religion" where both the belief and the functions are gone, and all that's left is a religion shaped hole that is filled with nihilism: the strong preying on the weak, self-indulgence, and an elite retreat into often paranoid fantasy.
These stages are shaped by the particular religion that disappeared, so the Zero Catholicisms aren't the same as Zero Protestantisms aren't the same as the Zero Islams. Science, being about what works rather than why you should be doing anything, simply didn't fill up these holes that once held morality and justification. For him, it seems, the Western world is primarily in a moral crisis, and we're seeing it in the mental decay of an elite that doesn't have to justify itself to anyone, ever (after religion has died.)
Personally, I can also see this in the deep desire of some people to obey AI, but I can't see it being fruitful at all. "Because the AI said so" is not particularly inspiring or ecstatic. It's just an extension of middle-class materialistic money as grace and job as devotion, which is notoriously unfulfilling. Will AI help you succeed if it can't tell you what it means to succeed?
The A.I. economic bubble is a mad scramble to ride the crest of a wave of stock-pumping expectations before the inevitable collapse and dump. Trillions of dollars of "value" from bloviating promises. It's worthy of a new chapter in Mackay's Extraordinary Popular Delusions and the Madness of Crowds. [1]
> religious impulses don't die with the withering of religion
Religions have of course come and gone throughout human history. The preceding deities, temples, and artwork are called mythology by people inside today's temples of fervour.
But let's be clear, disparate local tribal practices and beliefs are only formalised by a power structure for the masked purposes of the power structure.
What springs eternal is the maintenance of control in political and tribal hierarchies.
> A desire for a totalizing solution to all woes
The fact that our species exhibits astonishing credulity is illustrated throughout history to the present day, not just in religious activities but in every context of economic scams and demagoguery.
The thing about bubbles is that it's devilishly difficult to tell the difference between a total sham and a genuine regime shift while it's happening because the hype level is similar for both.
We should do things because they are hard, not because they are cheap and easy. AGI might be a fantasy but there are lots of interesting problems that block the path to AGI that might get solved anyway. The past three years we've seen enormous progress with AI. Including a lot of progress in making this stuff a lot less expensive, more efficient, etc. You can now run some of this stuff on a phone and it isn't terrible.
I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Data centers running on cheap clean power is entirely possible; and probably a lot cheaper long term. Kind of an obvious cost optimization to do. I'd prefer that to be sooner rather than later but it's nowhere near the highest priority thing to focus on when it comes to doing stuff about emissions.
> I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Oh no.
AI data centers are sucking up so much power it's making everyone's electric bill go up.
That's a tangible problem that dramatically impacts the poor and average person.
100 comments
[ 101 ms ] story [ 2046 ms ] threadAs a businessman, I want to make money. E.g. by automating away technologists and their pesky need for excellence and ethics.
On a less cynical note, I am not sure that selling quality is sustainable in the long term, because then you'd be selling less and earning less. You'd get outcompeted by cheap slop that's acceptable by the general population.
Now I run it through whisper in a couple minutes, give one quick pass to correct a few small hallucinations and misspellings, and I'm done.
There are big wins in AI. But those don't pump the bubble once they're solved.
And the thing that made Whisper more approachable for me was when someone spent the time to refine a great UI for it (MacWhisper).
It's better than Whisper, and faster, while running on CPU on my ten year old ThinkPad.
I had Claude make me Python bindings for it and add it to my voice typing app.
We live in the future.
Like it's not even clear if LLMs/Transformers are even theoretically capable of AGI, LeCun is famously sceptical of this.
I think we still lack decades of basic research before we can hope to build an AGI.
On the other hand, extracting usable insights from neuroscience? Not at all easy. Human brain does not yield itself to instrumentation.
If an average human had 1.5 Neuralink implants in his skull, and raw neural data was cheap and easy to source? You bet someone would try to use that for AI tech. As is? We're in the "bitter lesson" regime. We can't extract usable insights out of neuroscience fast enough for it to matter much.
Other energy usage figures, air pollution, gas turbines, CO2 emissions etc are fine - but if you complain about water usage I think it risks discrediting the rest of your argument.
(Aside from that I agree with most of this piece, the "AGI" thing is a huge distraction.)
UPDATE an hour after posting this: I may be making an ass of myself here in that I've been arguing in this thread about comparisons between data center usage and agricultural usage of water, but that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I still think the way these numbers are usually presented - as scary large "gallons of water" figures with no additional context to help people understand what that means - is an anti-pattern.
NYT article gift link where people reported wells ran dry after data centers moved in. : 'From Mexico to Ireland, Fury Mounts Over a Global A.I. Frenzy' https://www.nytimes.com/2025/10/20/technology/ai-data-center...
From https://www.eesi.org/articles/view/data-centers-and-water-co... , I understand there are two types of cooling with water in DCs, open-loop that's simple but water-intensive, and closed-loop that's expensive but efficient.
>> This can be achieved through air cooling using water evaporation, which is an open-loop and more water-intensive method, or through server liquid cooling.
https://andymasley.substack.com/p/the-ai-water-issue-is-fake
I also think the energy usage stuff is kind of nonsense. If energy usage is a major part of your operating expenses, you're probably going to locate your data center where energy is cheap, and cheap energy is always renewable. I'm sure you can find data centers that run off coal plants or other thermal power, but thermal power costs in the neighborhood of 100¢ per peak watt, while solar cells cost 12¢ per peak watt, so thermal power won't be competitive for very long.
I think you're still good on your original assertion, it seems many/most of the biggest players are using non potable in new facilities and also retrofitting some old ones to avoid potable water as well [1]
I think you'd be good either way: The distinction sounds like an important point until you realize that the cost of turning raw water potable is so vanishly small compared to the cost of these data centers. Some rough estimates place it as less than one single rack of a GB200 NVL72 to build enough-- or more economically, bolster the local existing plants for raw water processing. Even if they had to go to brackish water desalination the cost there looks to be mostly in ongoing electricity costs which amount to ~$3k per day such that their existing power plant build outs for these would easily cover it, or a few such new desalination plants to cover many many data centers.
I'm not unsympathetic to aspects of these overall concerns either, but critics have to do a lot better than concerns that are less hyperbolically expressed as the much less catchy "No AI!... without small and reasonable policies for covering proportional infrastructure cost increases!".
[1] https://datacentremagazine.com/articles/reclaimed-wastewater...
Key quote:
> If any region has a lot of freshwater and little potable water, the best way to make potable water more available and cheaper is to introduce a new large buyer, which will give the local utility enough revenue to upgrade and expand their treatment facilities. Saying that my data is misleading because Al "only uses valuable potable water" actually gets the issue backwards: adding demand for more potable water in regions with lots of freshwater makes potable water cheaper and more abundant for everyone else per unit.
This is a strawman. The big AI names aren't making a Pascal's wager type argument around AGI.
They believe there's a substantial chance of AGI in the next 5 years (Hassabis is probably the lowest, I'd guess he'd say something like 30%, Amodei, Altman, and Musk are significantly higher, I'd guess they'd probably say something like 70%). They'd all have much higher probabilities for 10 years (maybe over 90%).
You can disagree with them on probabilities. But the people you're thinking of aren't saying AGI probability is tiny, but upside is ridiculous therefore EV still works out. They're biting the bullet and saying probability is high.
Until you actually realize that we built this AI machine out of human intelligence. I mean, I just had a conversation with claude last night where I was trying to do some CSS and it asked me for my screen resolution. It made a passing comment saying "Your resolution is small? That's weird. Anyway..."
What are we doing here people? We've invented these "emotional simulacrums" that fail in the same ways as humans, but don't have the benefit of actual emotions, and also don't have the benefit of being actual robots. So worst of both worlds. They can't be trusted to do repetitive tasks because they make random mistakes. You can't trust them to be knowledgeable because they just invent facts. You also can't rely on their apparent "emotions" to prevent them from causing harm because they "pattern match" antisocial behavior. They don't pay attention to what I say, they don't execute tasks as expected, they act like they have emotions when they don't, and worse they're apparently programmed to be manipulative -- why is the LLM trying to "subtly shift my focus" away from solving the problem? That is worse than useless.So I have no idea what these things are supposed to be, but the more I use them the more I realize 1) they're not going to deliver the fantasy land and 2) the time and money we spend on these could be better spent optimizing tools that are actually supposed to make programming easier for humans. Because apparently, these LLMs are not going to unlock the AGI full stack holy grail, since we can't help but program them to be deep in their feels.
And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
Given this I am not certain we will get to AGI by simulating it in a GPU or TPU. We would need a new hardware paradigm.
I mean, you could argue that if you take into consideration all the generations (starting from the first amoeba) that it took to get to a standard human brain today, then the total energy used to "train" that brain is far greater. But I get your point and I do agree with you that our current hardware paradigm is probably not what's going to give us "god in a box".
We're totally incapable of building an AI that can do anything resembling that. We're still at the phase where robots walking on rough terrain without falling over remains a bit impressive.
I doubt the limitation is that we can't produce enough raw compute to replace a single bee.
if you put the brain in the shape of a tube you'd have a really long err, well, let's say it's not a good idea to do that. the brain gives me goosepimples, my brain too
Read: https://pythonic.ninja/blog/2025-11-15-ev-of-agi-for-western...
https://en.wikipedia.org/wiki/Demis_Hassabis
In the former case (charlatanism), it's basically marketing. Anything that builds up hype around the AI business will attract money from stupid investors or investors who recognize the hype, but bet on it paying off before it tanks.
In the latter case (incompetence), many people honestly don't know what it means to know something. They spend their entire lives this way. They honestly think that words like "emergence" bless intellectually vacuous and uninformed fascinations with the aura of Science!™. These kinds of people lack a true grasp of even basic notions like "language", an analysis of which already demonstrates the silliness of AI-as-intelligence.
Now, that doesn't mean that in the course of foolish pursuit, some useful or good things might not fall out as a side effect. That's no reason to pursue foolish things, but the point is that the presence of some accidental good fruits doesn't prove the legitimacy of the whole. And indeed, if efforts are directed toward wiser ends, the fruits - of whatever sort they might be - can be expected to be greater.
Talk of AGI is, frankly, just annoying and dumb, at least when it is used to mean bona fide intelligence or "superintelligence". Just hold your nose and take whatever gold there is in Egypt.
Yes, the huge expected value argument is basically just Pascal's wager, there is a cost on the environment, and OpenAI doesn't take good care of their human moderators. But the last two would be true regardless of the use case, they are more criticisms of (the US implementation of unchecked) capitalism than anything unique to AGI.
And as the author also argues very well, solving today's problems isn't why OpenAI was founded. As a private company they are free to pursue any (legal) goal. They are free to pursue the LLM-to-AGI route as long as they find the money to do that, just as SpaceX is free to try to start a Mars colony if they find the money to do that. There are enough other players in the space focused in the here and now. Those just don't manage to inspire as well as those with huge ambitions and consequently are much less prominent in public discourse
> LLMs-as-AGI fail on all three fronts. The computational profligacy of LLMs-as-AGI is dissatisfying, and the exploitation of data workers and the environment unacceptable.
It's a bit unsatisfying how the last paragraph only argues against the second and third points, but is missing an explanation on how LLMs fail at the first goal as was claimed. As far as I can tell, they are already quite effective and correct at what they do and will only get better with no skill ceiling in sight.
* AlphaFold - SotA protein folding
* AlphaEvolve + other stuff accelerating research mathematics: https://arxiv.org/abs/2511.02864
* "An AI system to help scientists write expert-level empirical software" - demonstrating SotA results for many kinds of scientific software
So what's the "fantasy" here, the actual lab delivering results or a sob story about "data workers" and water?
In the meantime, 100% agree, it's complete fantastical nonsense.
- Gödel-style incompleteness and the “stability paradox”
- Wolfram's principle - Principle of Computational Equivalence (PCE)
One of the red flags is human intelligence/brain itself. We have way more neurons than we are currently using. The limit to intelligence might very possibly be mathematical and adding neurons/transistors will not result in incremental intelligence.
The current LLMs will prove useful but since the models are out there, if this is a maxima, the ROI will be exactly 0.
People have been shitting on AGI since the term was invented by Ben Goertzel.
Anyone (like me) who has been around AGI longer than a few years is going to continue to keep our heads down and keep working. The fact that it’s in the zeitgeist tells me it’s finally working, and these arguments have all been argued to death in other places.
Yet we’re making regular progress towards it no matter what you want to think or believe
The measurable reality of machine dominance in actuation of physical labor is accelerating unabated.
An article in Illustrated London News, April 26, 1924 by G. K. Chesterton
If I understand his idea correctly, these societies that were developed with a religious justification, and a huge religious component, are of course losing it in the scientific age. The first stage they go through is "zombie religion" where people don't pretend to believe in the religion any more, but still insist that they share all of its values, and often become even more fanatical in the functions that the old religion served. The second stage is "zero religion" where both the belief and the functions are gone, and all that's left is a religion shaped hole that is filled with nihilism: the strong preying on the weak, self-indulgence, and an elite retreat into often paranoid fantasy.
These stages are shaped by the particular religion that disappeared, so the Zero Catholicisms aren't the same as Zero Protestantisms aren't the same as the Zero Islams. Science, being about what works rather than why you should be doing anything, simply didn't fill up these holes that once held morality and justification. For him, it seems, the Western world is primarily in a moral crisis, and we're seeing it in the mental decay of an elite that doesn't have to justify itself to anyone, ever (after religion has died.)
Personally, I can also see this in the deep desire of some people to obey AI, but I can't see it being fruitful at all. "Because the AI said so" is not particularly inspiring or ecstatic. It's just an extension of middle-class materialistic money as grace and job as devotion, which is notoriously unfulfilling. Will AI help you succeed if it can't tell you what it means to succeed?
> religious impulses don't die with the withering of religion
Religions have of course come and gone throughout human history. The preceding deities, temples, and artwork are called mythology by people inside today's temples of fervour.
But let's be clear, disparate local tribal practices and beliefs are only formalised by a power structure for the masked purposes of the power structure.
What springs eternal is the maintenance of control in political and tribal hierarchies.
> A desire for a totalizing solution to all woes
The fact that our species exhibits astonishing credulity is illustrated throughout history to the present day, not just in religious activities but in every context of economic scams and demagoguery.
[1] _ https://en.wikipedia.org/wiki/Extraordinary_Popular_Delusion...
I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Data centers running on cheap clean power is entirely possible; and probably a lot cheaper long term. Kind of an obvious cost optimization to do. I'd prefer that to be sooner rather than later but it's nowhere near the highest priority thing to focus on when it comes to doing stuff about emissions.
Oh no.
AI data centers are sucking up so much power it's making everyone's electric bill go up.
That's a tangible problem that dramatically impacts the poor and average person.