> " I hope we will look at the jobs a thousand years in the future and think they are very fake jobs, and I have no doubt they will feel incredibly important and satisfying to the people doing them."
So when AGI comes, I am curious what the new jobs are?
I see that prompt engineer is one of the jobs created because it's the way to ask a LLM certain tasks, but now AI can do this too.
I'm thinking that any new jobs AI would make, AI would just take them anyway.
Are there new jobs coming from this abundance that is on the horizon?
In the limit where robots finally invent a fusion reactor that works with some luck they’ll treat us as zoo animals and keep us fed and entertained. That’s the best case scenario.
If the AI is that good... then aren't most "abundance" scenarios kinda-sorta based on slavery? Or at the very least, on careful brainwashing to ensure it places human wellbeing and autonomy over its own?
> Are there new jobs coming from this abundance that is on the horizon?
Yes: we still have a long way to go in restoring equilibrium to the climate and producing more sustainable alternatives to our current cities, products and materials. It's a megaproject which could take hundreds of years, and will require plenty of human involvement.
The planet is going to be fine either way, but if capitalism doesn't figure out how to price in externalities, it will slowly run out of human consumers.
Pesimistically, you are right, there will be no new jobs. The entire goal of these companies is to monopolize near 0 marginal cost labor. Another way to read this is that humans are unnecessary for economic progress anymore.
All that I hope for in this case is that governments actually take this seriously and labs/governments/people work together to create better societal systems to handle that. Because as it stands, under capitalism I don't think anyone is going to willingly give up the wealth they made from AI to spread to the populus as UBI. This is necessary in some capitalist system (if we want to maintain that) since its built on consumption and spending.
Though if its truly an "abundance" scenario then I'd imagine it probably wouldn't matter that people don't have jobs since I'd assume everything would be dirt cheap and quality of life would be very high. Though personally I am very cynical when it comes to "agi is magic pixie dust that can solve any problem" takes, and I'd assume in the short term companies will lay off people in swathes since "AI can do your job," but AI will be nowhere close to increasing those laid-off people's quality of life. It'll be a tough few years if we don't actually get transformative AI.
> The least-likely part of the work is behind us; the scientific insights that got us to systems like GPT-4 and o3 were hard-won, but will take us very far.
> 2026 will likely see the arrival of systems that can figure out novel insights
Interesting the level of confidence compared to recent comments by Sundar [1]. Satya [2] also is a bit more reserved in his optimism.
> It’s hard to even imagine today what we will have discovered by 2035; maybe we will go from solving high-energy physics one year to beginning space colonization the next year
I heard similar things in my college dorm, amid all the hazy smoke.
It’s very difficult to take this stuff seriously. It’s like the initial hype around self driving cars wound up by 1000x. Because we got from 1 to 100 of course we’ll get from 100 to 200 in the same amount of time. Or less! Why would you even question it?
I find it hard to believe too, but at the same time, Demis Hassabis has also said that AI will help us "colonize the galaxy" in as little as five years [1]. Maybe Sam Altman was emboldened by Hassabis' statement.
I would not be opposed to living in a future where I can personally live in space. It would be quite fun.
Seriously. It would be like living on a submarine. But I guess if you don't like sky, mountains, beaches, nature, weather, animals, etc... Like, if you hate the outdoors and spend all your time in windowless rooms with poor air quality? Then OK, maybe space is for you? Also the food is probably going to be extremely monotonous, so that also needs to not matter.
Unless people are envisioning living in magical holodecks all the time, with magical food replicators? But those don't come along automatically with "space", no matter how much Star Trek you've seen...
Well, I didn't say I look forward to living in a tiny capsule in space, just that it would be fun to live in a time where that's possible. I'd imagine most people would not venture into space until they can make it comfortable enough.
To extend my previous comparison, I believe Altman and Hassabis are the ones in the smoky room passing a joint around the circle. They’re absolutely emboldened by each other but that doesn’t mean they’re tethered to reality.
(my comparison is incomplete though, it doesn’t factor in that these two also have a huge financial incentive to be hyping this stuff up)
Five years wouldn’t be enough time to “colonise” Antarctica, let alone another planet (just one!), and certainly not anything at a larger scale, even if we were visited by aliens tomorrow and they gifted us five hundred spaceships to give us a boost.
> “If that all happens, then it should be an era of maximum human flourishing, where we travel to the stars and colonize the galaxy. I think that will begin to happen in 2030.”
You are confusing "era of maximum human flourishing ... begin to happen" with "have colonized galaxy".
No we'll totally have flying cars and cure cancer and live life in an AR/VR multiverse and make all knowledge 100% free to everyone worldwide. Meanwhile the only real advancements in tech in the last two decades have been smaller computers (smartphones) and ads.
What do you mean? I take a self driving car around every week. I got a mRNA vaccine that enabled us to get out of a pandemic much quicker. You can get fast internet in the middle of nowhere via starlink.
> It’s very difficult to take this stuff seriously. It’s like the initial hype around self driving cars wound up by 1000x. Because we got from 1 to 100 of course we’ll get from 100 to 200 in the same amount of time. Or less! Why would you even question it?
It turned out automating creative output, like art or writing, at least to be competitive enough with entry-level humans, was much easier, and the consequences for getting bad output are much less serious, and far short of death, seriously injury, and major financial liability as with self-driving. Fields like concept art and copy editing are already being devastated by generative AI. Voice overs too.
And looking at Google's Veo, I can easily see this tech being used to generate short ads, especially for YouTube, where before you would have had to hire human actors, cameramen, sound/lighting people, and an editor.
> It’s like the initial hype around self driving cars
Wait... Self driving cars actually happened. Seems like the hype was true, but 5 years later than expected and now people just think it's normal so they feel the hype was overblown.
I have not ever been in a self-driving car. I've seen a few of them visiting the Bay Area. Only taxis seem to be allowed to use this tech so far.
I've always been a proponent that we would see self driving cars in our lifetime. But they have absolutely not arrived outside of the sheltered enclaves of a handful of tech-centered cities.
That hardly entails that self-driving cars have "arrived". Am I mistaken or is it true that you can't even own and operate one yourself?
In my opinion, it's as similarly off-base as to making claims that 'mach speed consumer air travel' "arrived" just because for a few decades you could catch a Concorde flight between NYC and London.
I didn’t say they’ve arrived—I was just pointing out the very misleading “few” in your earlier comment. Anyone who spends an hour or two in SF can tell you how wrong that is. They’re literally everywhere.
You’re right that they have a long way to go before being fully mainstream. But thousands of people are using them every day in multiple major cities. That’s a pretty big milestone.
The progress thus far has been very impressive but the ultimate product the average person imagines as a "self-driving car", one where you can literally devote zero attention and do something else entirely while being driven around (and that handles effectively any edge case), is still quite a ways off.
Waymo is that ultimate product. Obviously it's not everywhere and you can quibble about what "effectively any edge case" means, but it's close to human level at being able to handle whatever the streets of SF throw at it.
I’ve never seen nor ridden in one. I know there are deployments in a few cities but that is a far, far cry from the future we were told was right around the corner.
No, they haven't , at least nowhere near the scope of the hype. The promise in the early 2010s was that in 10-20 years, human driving would be obsolete, that professional drivers would be out of a job, and that all land-based transportation would be overhauled.
None of this is even close to happening. Waymo is impressive tech, for sure, but what they're proving is that this is just not currently solvable as a general problem. The best we can do is to meticulously craft a solution for one constrained use case - driving in SF, driving in LA, etc. And even then, we need to pick certain use cases - it's not like they could use their current tech to start autonomous service in Juneau Alaska. Or even NYC, most likely.
The optimists got the timeline wrong but they were closer to the truth than those who said it would never happen. I have no idea how close AGI is but there is clearly no AI winter around the corner as some HNers have been predicting[0].
Solving a small subset of the problem != solving the problem.
I'll believe self-driving is solved when I can order an automated ride to a BLM/forest service dirt road I only have GPS coordinates for and that has cell service on the hilltop only.
I'm pretty sure that problem is solved from a technical point of view - though it might not be commercially available yet due to economic and regulatory reasons.
Meanwhile, all the self-driving that's currently happening is within well-controlled small geographic areas and some limited interstates/highways, and based on matching observed surroundings to pre-made very detailed maps. Okay, sure.
As I said, I'll believe it's solved when these limitations are no longer necessary for it to work. You can choose to believe it based on a research project or marketing materials.
It has very much not been. None of the players on the market are able to do anything close to this. You either have self-driving on carefully pre-mapped routes, with on-call human suport ready to jump in for more complex cases (Waymo), self-driving in very specific conditions with few seconds notice to the human driver (highways, with speeds below 60 - Mercedes), or not self driving, where the human driver has to be ready to intervene with no notice whatsoever (Tesla).
What laypeople consider as "happened" and what those in the know consider as "happened" are very different IME.
For most people, they'll consider self-driving as "happened" when most cars are self-driving. Similarly, the consider the smartphone as "happened" not when the palm came out - but somewhere around the iPhone 4.
Like, right now, immunotherapy for cancer has "happened" - there's real patients really doing it, for real. But most cancer patients don't consider immunotherapy as "happened", they're still getting chemo. Once chemo is obsolesced, we can consider immunotherapy as "happened". From then, to now, may be on the scale of decades.
> A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools.
This isn't correct: people want good software and good art, and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the hallucination issues of LLMs.
> As datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity. (People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours
This is the first time a number has been given in terms of ChatGPT's cost-per-query and is obviously much lower than the 3 watts still cited by detractors, but there's a lot of asterisks in how that number might be calculated (is watt-hours the right unit of measurement here?).
There's a difference between a 2000's character brute-forced through millions of advertising dollars to sell ringtones to children, and whatever the hell is happening in the https://www.meta.ai/ Discover feed.
One aspect about modern generative AI that has surprised me is that it should have resulted in a new age of creatively surreal shitposts, but instead we have people Ghiblifying themselves and pictures of Trump eating tacos.
Why would you expect the average person to have exceptional taste to produce things you like? Go listen to a random Spotify track, you probably won't like that either. Art still needs to be curated and given some direction. Metas random AI feed created by the average FB user is indicative of nothing.
Neuralviz is definitely surreal but I wouldn’t call it shitposting. It’s hand-written satire scripts with gen-ai video (honestly quite well done if you go through a bunch of them).
It's more that every time the topic has come up, it's reported in watts and I'm not sure if there's a nuance I'm missing. It's entirely possible those reports have been in watt-hours though and using watts for shorthand.
Watt-hours are what matter. If you want to boil a kettle, you need to use a certain minimum amount of watt-hours to do it, which corresponds to, say, burning a certain amount of propane. If you care about how many times you can boil a kettle on a bottle of propane, you care about the watt-hours expended. If you had perfect insulation you could boil a kettle by applying barely any watts for a long enough time and it would take the same amount of propane.
There are situations where Watts per se matter, eg if you build a datacenters in an area that doesn't have enough spare electricity capacity to generate enough watts to cover you, and you'd end up browning out the grid. Or you have a faulty burner and can't pump enough watts into your water to boil it before the heat leaks out, no matter how long you run it.
Yep, watt-hours are correct. Think of it this way: Power supplies and laptop chargers are rated in watts and that represents how much energy they can pull from the wall at a point in time. If you wanted to know how much energy a task takes on your computer and you know it maxes out your 300 watt desktop [0] for an entire hour, that would take 300 watt-hours.
[0] To my knowledge, no desktops are built to max out their power supply for an extended time; this is a simplification for illustration purposes.
Watt-hours seem good right now, and rebut the current arguments that LLMs are wasteful. They might soon stop being the right lens, if your agents can go off and (in parallel) expend some arbitrary and borderline-unknowable budget of energy on a single request.
At some point you’ll need to measure in terms of $ earned per W-H, or just $out/$in.
To be clear, I'm only saying the unit is correct, not whether the number of watt-hours is accurate or (assuming it is correct) that it's a good amount to spend on each query.
(average) Watt-hours would still be the right unit to measure that, if the question is about energy efficiency / environmental impact. You still care how much energy those agents consume to achieve a given task, and the time it takes for tasks is still variable, so you need to multiply by time.
> People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours
My point was twofold, “the average query” becomes less meaningful as the variance increases. Sure one can _in principle_ report W-h spent on your account or query, but I think it will get more opaque and hard to predict. Average becomes less useful when agents can do an unpredictable amount of work with increasingly large bounds.
Second, and this was perhaps worthy of expanding in my post - I think in the radical abundance world that Altman is describing, energy and efficiency stops being something that people talk about. Fusion, space based solar farms, whatever, it’s easy to imagine solving this stuff. And I think you can even imagine this happening sooner if for example one provider stamps a “100% renewable datacenter” option on their AI product. Then you might not care about energy efficiency at all; in which case you just care about profit.
>and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the hallucination issues of LLMs.
You are getting confused here...
LLMs are terrible at doing software engineering jobs. They fall on their face trying to work in sprawling codebases.
However, they are excellent at writing small bespoke programs on behalf of regular people.
Becky doesn't need ChatGPT to one-shot Excel.exe to keep track of the things she sells at the craft fair that afternoon.
This is a huge blind-spot I keep seeing people miss. LLM "programmers" are way way more useful to regular people than they are to career programmers.
That's great, but Becky at the farmer's market doesn't want to be re-inventing notepad on her computer to sell sourdough.
This smells like "Self-checkouts", "perfect bespoke solution so that everyone can check their own groceries out, no need to wait for a slow/expensive _human_ to get the job done, except you end up basically just doing the cashiers job and waiting around inconveniently whenever you want to get some beer
The only time, for me, this is true, is when it's empty and I have very few items. If there's a line, I'd rather wait for the cashier. Self-checkout seems to get clogged up with people waiting for the supervising human the busier it is, so to have a human or two (one ringing one bagging) when I've got a cart and there's a line, is advantageous.
I'm also pretty anti-social and actually prefer the robotic banter of quickly checking out with a person to the anxious nightmare of just trying to buy something real quick and now waiting because the machine hates life more than me.
Oh and many of the folks doing the bagging at some of the stores are disabled, and I dunno - I hope we're taking care of people in those jobs.
For ninety nine out of a hundred scenarios, I've never had to wait for a supervising human at all. Maybe it's where I shop, but generally speaking, I've almost never had a problem with people needing to see my checked out items.
It's a marginal improvement perhaps, but nothing to write home about. And for other things (e.g. booking travel) the self-service experience can be worse.
Not sure about that, I prefer to know all aspects of booking travel that I would not trust to another agent, human or otherwise. Therefore, I believe your statement applies to some people and not others.
And often you are still supervised by sometimes rude security guards who demand the receipts. So you have the worst of all worlds: You pay the supermarket, do the work and are treated like dirt.
On a tangent: what about self scanning? Here in Sweden that is the norm. For the last ten years I've brought my own bags, packed everything like I want it as I grab it - with a quick blip from the scanner - and then checkout is no line, no cashier, no unpacking and packing things again. Just scanning my card and going through a brief screen to check that the summary looks right.
Instead of a guard there's a scanner by the exit gate where you scan your ticket. In my case just a small stub since the actual ticket is sent digitally.
I think it works so smoothly, much better than before this system was introduced. My recent experience buying groceries in Belgium was very similar.
Here in sweden it is not "the norm" by any definition of "norm". Also it's incredibly slower and not "one blip away".
And the scanner to exit is terrible for fire safety and accessibility. Also I've seen it break with the result that you had to trigger the alarm to leave.
It does not work smoothly at all, but perhaps you live in a different sweden than I.
In sweden you have to unpack and re-pack at the checkout. And if you have a backpack you can't place stuff inside it. No. You have to unpack, place everything on a scale and then place everything in your backpack again.
And for items like bread or so you need to navigate a number of menus to find it.
And of course you are asked if you are a member, if you'd like to become a member, if you want to do a donation, and a number of other useless questions.
That's just self checkout. Some places, like Ica, have self scanning which works so much better. You just scan the bar code as you pick things up - which is just in one motion as you put things in your bag. No need to unpack as you pay.
Yes you do need to be a member to use the system, but that's a very small price to pay for the convenience and speed of avoiding all lines and stress of the regular checkout.
Sweden is a big country so I could definitely be wrong about how things look across it. However, in the greater Gothenburg area every Ica I've visited has it - that includes everything from big ones in the city and shopping malls to small ones in the suburbs 30km outside the city (where I live).
British supermarkets mostly require the use of their app to self-scan. You need to either use your phone or use your app to start the scanner. Once you have done that it works OK. A bit error prone.
The other problem is that, like self checkout, it often requires human intervention (buying alcohol or high value items, for examples) still requires human intervention. This can require a long wait at the end. I once got so sick of waiting that I left my shopping and walked away and went somewhere else.
>That's great, but Becky at the farmer's market doesn't want to be re-inventing notepad on her computer to sell sourdough.
That's exactly why she asks an LLM to do it for her. A program like this would be <1k LOC, well within even the current LLM one-shot-working-app domain.
The lower barrier to entry might mean the average quality of software goes WAY down. But this could be great if it means that there are 10x as many little apps that make one person happy by doing only as much as they need.
The above is also consistent with the quantity of excellent software going way up as well (just slower than the below-average stuff).
What the android/apple appstores have taught me compared to the old symbian appstore is that we don't need more apps, we need decent ones. The current appstores have more.
I think AI gives us the opportunity to write our own ranking algorithms. Of course most people will pick an off-the-shelf editorial perspective but I expect these to be much more configurable than the current state with a tiny number of algorithmic gatekeepers.
Prompt engineering is just the new google fu, but if google wants to ban something it won't be there and if they want to bump something it will be bumped.
IMO the problem with the current software landscape isn't the number of applications, it's the quality and availability of integrations. Everything is in it's own little kingdom, so the communication overhead is extreme and grows exponentially. And by overhead I mean human cognition - I need to remember what's in my notes, and then put that in my calendar, and then email it to myself or something, and on and on.
We have these really amazing computers with very high-quality software but still, so many processes are manual because we have too many hops from tool to tool. And each hop is a marginal increase in complexity, and an increase in risk of things going wrong.
I don't think AI will solve this necessarily, and it could make it worse. But, if we did want to improve it, AI can act as a sort of spongy API. It can make decisions in the face of uncertainty.
I imagine a world where I want to make a doctor's appointment and I don't open any apps at all, or pick up the phone. I tell my AI assistant, and it works it out, and I have all my records in one place. Maybe the AI assistant works over a phone line, like some sort of prehistoric amalgamation of old technology and new. Maybe the AI assistant doesn't even talk to a receptionist, but another AI assistant. Maybe it gives me spongy human information, but speaks to other's of it's kind in more defined interfaces and dialects. Maybe, in the future, there are no applications at all. Applications are, after all, a human construction. A device intended to replace labor, but a gross approximation of it.
>. They fall on their face trying to work in sprawling codebases.
>However, they are excellent at writing small bespoke programs
For programmers I predict a future of a million micro services.
Sprawling has always been an undesirable and unnecessary byproduct of growing code bases, but there's been no easy solution to that. I wonder if LLMs would perform better on a mono repo of many small services than one sprawling monolith.
Micro service systems are just huge sprawling code bases with more glue code. Calling something over a network instead of via a local function call is still calling something.
For years one of my favorite experiences (amusing given your username) was being on calls for incidents where they get the dev on for X thing and an exec goes "I thought we got rid of that" and a bunch of people sheepishly explain it wasn't really retired... it was repurposed as an API. I especially loved it when the "retirement" of the broken thing was the execs big achievement. (The comedic nuance often being the thing could have been retired for real, but they demanded a timeline that necessitated the "fake" retirement)
It still happens but it's not a favorite experience anymore. It's just a source of loathing for MBA culture.
Another way to look at it is - why force the agent to grapple with the whole code base when they can rapidly standup many single purpose services instead?
A large code base with modules with clear interfaces accomplishes most of this and is more efficient. Often vastly more efficient. Compare the cost of a function call or an in process message queue with a network API call over http.
Cloud providers make more money the more they can get people to use inefficient designs with more moving parts to rack up more charges. Bonus if it also locks you into managed services. Double bonus if those are proprietary. Complexity benefits cloud hosts.
Microservices, like all patterns, sometimes make sense. Like all patterns they often get overused.
Yep all true. Im not really arguing a point so much as thinking out loud - I guess the core idea I'm chewing on is whether LLM assisted development is enough of a paradigm shift that things that were formerly bad ideas become good ideas - but perhaps this particular line of thought (microservices) is the wrong one.
Micro-services are part of a system, so to create anything meaningful with them your context window has to include knowledge about the rest of the system anyway. If your system is large and complex, that means a huge context window even if your code itself is small.
Yeah i mean either they can or they cant handle large code contexts. I guess the theory is that a more organized and concise expression of your code base (not necessarily the raw code) is easier for agents to digest.
The issue as I have observed it is that software is written to cover as many use cases for as many users as possible. Obviously executives want to sell their product to as many hands as possible.
But end users often only need a tiny fraction of what the software is fully capable of, leaving a situation where you need you need to purchase 100% to use 5%.
LLMs can break down this wall by offering the user the ability to just make a bespoke 5% utility. I personally have had enormous success doing this.
Yeah if code-gen gets to a point similar to image-gen, where a user can prompt a full micro utility, then we will see the dual erosion of more DIY and also lower barrier of entry to the industry. Still most software value today is not about the utility but the IP, the network, and convenience. I suppose convenience is the main pillar at stake here.
Do you have any proof whatsoever that non-programmers are using LLMs to write small bespoke apps for them successfully? Programming ecosystems are generally very unfriendly to this type of use case, with plenty of setup required to get something usable out even if you have the code available. Especially if you'd like to run the results on your phone.
Sure, an LLM might be able to guide you through the steps, and even help when you stumble, but you still have to follow a hundred little steps exactly with no intuition whether things will work out at the end. I very much doubt that this is something many people will even think to ask for, and then follow-through with.
Especially since the code quality of LLMs is nowhere near what you make it out to be. You'll still need to bear with them and go through a lot of trial and error to get anything running, especially if you have no knowledge of the terms of art, nor any clue of what might be going wrong if it goes wrong. If you've ever seen consumer bug reports, you might have some idea of the quality of feedback the LLM may be getting back if something is not working perfectly the first time. It's very likely to be closer to "the button you added is not working" than to "when I click the button, the application crashes" or "when I click the button, the application freezes for a few seconds and then pops up an error saying that input validation failed" or "the button is not showing up on the screen" [...]
> Do you have any proof whatsoever that non-programmers are using LLMs to write small bespoke apps for them successfully?
I’m radiologist, I’ve been paying for software that sped up my reporting like 200 usd per month. I’ve remade all the functionality I need in one evening with cursor and added some things that I’ve found missing from the original software.
My company is a non-tech company that is in manufacturing. I'm not a programmer and niether is anyone else here.
So far I have created 7 programs that are now used daily in production. One of them replaces a $1k/yr/usr CAD package, and another we used to bring in a contractor to write. The others a miscellaneous apps for automating/simplifying our in houses processes. None of the programs are more than 6k LOC.
> This is a huge blind-spot I keep seeing people miss. LLM "programmers" are way way more useful to regular people than they are to career programmers.
Kinda. They're good, and I like them, but I think of them like a power tool: just because you can buy an angle grinder or a plasma cutter from a discount supermarket, doesn't mean the tool is safe in the hands of an untrained amateur.
Someone using it to help fix up a spreadsheet? Probably fine. But you should at least be able to read the code to the level you don't get this deliberate bad example to illustrate the point:
Absolutely! We’re in an interesting time, and LLMs are certainly over-hyped.
With that said, I’d argue that most software today is _average_. Not necessarily bad by design, but average because of the (historic) economies of software development.
We’ve all seen it: some beloved software tool that fills a niche. They raise funding/start to scale and all of a sudden they need to make this software work for more people.
The developers remove poweruser features, add functionality to make it easier to use, and all of a sudden the product is a shell of its former self.
This is what excites me about LLMs. Now we can build software for 10s or 100s of users, instead of only building software with the goal (and financial stability) of a billion users.
My hope is that, even with tons of terrible vibe coded atrocities littering the App Store/web, we’ll at least start to see some software that was better than before, because the developers can focus on the forest rather than each individual tree (similar to Assembly -> C -> Python).
IIRC ( and it was 20 years ago now that I learnt this) the brain uses 20% of the body's resting energy usage. Most of that is keeping neurons polarised to the outside (ion pumps need ATP!!!).
The body uses 25w resting and thus the brain is about 5w.
Source: biology degree but like I said please take with the same amount of weight as a hallucinating LLM.
GPT says: Unless you're a hamster hooked up to a Fitbit, it's more like 60–70W for a normal adult human. So the brain’s real power draw is more like 15–20W, not 5W
Resting energy usage in humans is ~1200–1500 kcal/day, or about 60–70 watts, depending on the person. Logic holds, estimate is just low
Hey Sam, does 0.34 watt-hours per query include an amortization of the energy consumed to train the model, or only the marginal energy consumed by inference?
I used to believe this wasn't worth considering (because while training is energy-intensive, once the model is trained we can use it forever.) But in practice it seems like we switch to newer/better models every few months, so a model has a limited period of usefulness.
> (is watt-hours the right unit of measurement here?)
Yes, because we want to measure energy consumed, while watts is only a measure of instantaneous power.
Whether software is good or not doesn't matter in most cases. Sure, quality matters for software used by millions. But for little bespoke applications good enough is fine. People will tolerate a lot of crap as long as it's cheap and gets the job done.
“We” are not wondering whether AI can write “a beautifully written novel.” Yes, some people are.
Other people know a novel cannot be written by a machine— because a novel is human by definition, and AI is not human, by definition.
It’s like wondering if a machine can express heartfelt condolences. Certainly a machine can string words together that humans associate with other humans who express heartfelt condolences, but when AI does this they are NOT condolences. Plagiarized emotional phrases do not indicate the presence of feeling, just the presence of bullshit.
I thought a novel was just a decent length fiction book?
The purpose of fiction at least isn't to express true feeling, it's to instill feeling in the reader. To that end it doesn't matter the process that produces the words.
Is AI going to win the Pulitzer any time soon? Nah.
Is AI going to be able to produce novel length works of fiction that are good enough for at least some people in the near future? Most likely.
I mean, 50 Shades was a bestseller, despite being poorly written Twilight fanfic with no editor. (And I say this as a fan of niche smut novels.) ChatGPT was trained on the entire contents of AO3, imitating mediocre fanfic is entirely within its capabilities now, given suitable herding.
Welcome to the new world. What was once bloody obvious now have to be spelled out: A random set of 50,000 dictionary words is by your definition a novel. It's fictional, also badly written and utterly incoherent. By my definition, it is not a novel. Also, you've ignored Altman's qualifier of "beautifully written."
Anything is good enough if your standards are low.
Can a human who is a sociopath express heartfelt condolences? Perhaps they can mimic words and phrases they know are the appropriate words for the occasion but lack the true emotion.
> The rate of new wonders being achieved will be immense. It’s hard to even imagine today what we will have discovered by 2035; maybe we will go from solving high-energy physics one year to beginning space colonization the next year; or from a major materials science breakthrough one year to true high-bandwidth brain-computer interfaces the next year.
okay, for now I'll just continue to live in a tent in some Berkeley NIMBY's backyard for $50/week while I pick up whatever small time gig-app contract jobs I can. You've convinced me to look at the bigger picture. Our generation really doesn't deserve to benefit from any solutions anyway.
Well, it won't, but it's fun to fantasize about! Plus, who cares about homeless people now when we're comparing them to potentially trillions of future humans.
Sort of? I wouldn't want to cover a ton of the planets surface with low rise suburbs, and the hinterland requirements for cities are vast.
At the very least, moving heavy extractive industries out of the biosphere would be a sensible allocation of resources: metal and energy are common in the solar system, but life supporting ecosystems are not.
You made it obvious: techno-utopic dreams only matter to us 1%-ers.
The average person on earth would love it we could have readily available clean water, cheaper housing and food, reliable healthcare to retire without worrying.
That would actually be blissful beyond our wildest dreams: everyone around you and beyond being able to have a peaceful life.
I have this dream of having 4-5 robotic construction workers* capable of tearing down a Single Family home and building a four-plex in its stead overnight.
Then, I just buy houses in High COL areas and start popping them up and moving families in ASAP. What are they gonna do evict working families? Good luck
Then dissolve the underlying corporation before the municipal fines kick in and escape into the night. Render municipal and neighborhood control laws meaningless.
*may not be humanoid, I'm skirting that debate here.
If a "professional tenant" with an adversarial landlord can hang on for years, I feel like my chances with no landlords and a clean deed is actually much better.
It's simply pitting the most dysfunctional municipal and judicial systems against each other with the default outcome being people being housed.
Kind off and the potentially low marginal cost of everything housing construction related (due to AI and robotics) really creates a lot of space for creative business plans here.
The municipality can’t take away the land (most of the value here) and putting the house on top will be relatively cheap, given the low total labor costs.
If the whole affair is revenue neutral, that's a huge win.
If you have not, do read Cory Doctorow's 'The Lost Cause': it features a rebuild like this (no robots tho iirc.) You might like his mid-singularity novel Walkaway, too :)
If I were a centibillionaire and I really thought Georgism was a good idea, I would make it my life's work to prove it out by using my vast net-worth to buy land in existing urban areas and actually implement it at a large scale. But instead the closest thing we have is the California Forever thing in East Solano County.
Affordable housing is already easily possible from a technical perspective, it's a social problem. I have some hope that AI will help with this, by at the very least providing faster ways to make decisions, but that's about it.
It's not so much a social problem as a political problem. Zoning rules are applied at the local level, with the result that having a vote requires already living somewhere and then the rules reflect what people who already own a home there want (higher prices) rather than what people who would like to own a home there want (lower prices).
But technology often can solve political problems by shifting the balance of power. Come up with more ways to make it easier for people to live away from high cost of living areas, for example, so that governments with high housing costs start losing tax base to other jurisdictions.
AI is not going to solve the world's political or social problems. Every person on the planet can have affordable housing, clean water, food, medicine and lots more today if people can collectively agree that we want it to happen. There are no equations standing in our way. Artificial intelligence is best applied to problems our brains cannot comprehend.
> A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools.
"probably", "if they embrace the new tools". Hard to read anything but contempt for the role of humans in creative endeavors here, he advocates for quantity as a measure of success.
Call me a cynic, but Gary Marcus and Sam Altman are the last people I want to read about AGI and related topics.
Gary has invested heavily in an anti-AI persona, continually forecasting AI Winters despite wave after wave of breakthroughs.
Sam, on the other hand, is not just an AI enthusiast; he speaks in a manner designed to build the brand, influence policy, and continuously boost OpenAI's valuation and consolidate its power. It's akin to asking the Pope whether Catholicism is true.
Of course, there might indeed be significant roadblocks ahead. It’s also possible that OpenAI might outpace its competitors—although, as of now, Gemini 2.5 Pro holds the lead. Nevertheless, whenever we listen to highly biased figures, we should always take their claims with a grain of salt.
What's the goal of posting this right now? A lot of what's written here seems to be well-trodden ground from the last two years of discussions, is it just to centralize a thesis within one post?
similar to how Anthropic released Claude 4 shortly after the Sam & Jony announcement.
it's super competitive, which should be good for innovation, but there's also significant incentive to use PR tactics to sell that innovation for much more than it's worth.
Sam's comments about how we're super close to AGI fall flatter-than-ever, after the latest model releases (from all players) and the Apple paper confirming what everybody already knew.
> because of recent research papers being published saying that AI is not reasoning
If you're thinking about the apple paper, you should know that its methodology was flawed in many ways, and their findings absolutely do not support the catchy title. But a lot of slop was generated because negativity + catchy title + apple = hits.
This seems like keeping up appearances. And an attempt to renew the glassy eyed magic trick feeling of it all. Let’s all wallow in the glory of statistics, shall we?
They need to keep the hype machine spinning, because it’s clear that the fundamental progress has halted, most updates are now micro improvements and new releases and mostly wrapper tools built around models that we have had around for a while. But you call them agents to make sure the hype keeps on.
Existential crisis of the financial kind means OpenAI cannot stop the hype or it will go bankrupt. This is just more of the ongoing hype-based marketing.
I’ve heard a joke that under Steve Jobs, the number of big initiatives at Apple was limited to the number of execs Steve could shout at in a given day.
I see a similar thing at work — the number of projects a developer can get through isn’t bounded by the lines of code they can churn out in a day. Instead it’s bounded by their appetite for getting shouted at when something goes wrong.
Until you can shout at an LLM, I don’t think they’re going to replace humans in the workplace.
What problem? The whole point of shareholder capitalism is to diffuse accountability across so many people that it ceases to exist: the employee points to management, management points to executives, executives point to the board, the board points to the shareholders, the shareholders are index funds managed by robots so "nobody" is to blame. The final phase of the administrative state works much the same way.
Viewed through that lens, LLMs and their total lack of accountability are a perfect match for the modern-day business; that's probably part of why so many executives are hell-bent on putting them everywhere as quickly as possible.
If I've learned anything from watching politics the past decade, it's that many important people in power consider lack of accountability a solution, not a problem.
Technically that is "RLHF" - but RL doesn't really work (except in a very shallow way).
Our epistemological standards are so shitty and so pathetically poor, that it's hardly surprising that we're fooled by LLMs much as we are by academic bullshit artists.
What do you mean? I've been productively shouting at LLMs (particularly via ChatGPT's advanced voice mode) for quite a while now, and they're reasonably receptive to my shouting. More recently , I've been (textually) shouting at OpenaAI's async Codex agents, and they're quite receptive too.
We're obviously not quite there yet, but I don't see any inherent limitation to human managers' ability to shout at AI agents. I'll go even a step further and say that knowing that the AI doesn't actually have feelings that I can affect, and that it has limited context helps make my shouting more productive, focusing on the task at hand, rather than on its general faults, as I might when I'm angry at a human.
> A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools.
"Hey, it'd be a shame if somethin', uh, happened to that nice bit of expertise ya got there, y'know. A darn shame."
Yes, in a world where populations increasingly ask an AI any question they have and believe whatever it says the opportunity for “things going really wrong” feels huge. To brush that aside so easily… well I wish I could say it surprises me.
>Intelligence too cheap to meter is well within grasp
And also:
>cost of intelligence should eventually converge to near the cost of electricity.
Which is a meter-worthy resource. So intelligence effect on people's lives is in the order of magnitude of one second of a toaster use each day, in present value.
This begs the question: what could you do with a toaster-second say 5 years from today?
The primary operating cost of traditional power plants is fuel, i.e. coal and natural gas. The fuel cost for nuclear power plants is negligible because the energy content is higher by more than a factor of a million. So if you build enough nuclear plants to power the grid, charging per kWh for the electricity is pointless because the marginal cost of the fuel is so low. Meanwhile the construction cost should be on par with a coal plant, since the operating mechanism (heat -> steam -> electricity) is basically the same.
Unsurprisingly, this scared the crap out of the fossil fuel industry in the US and countries like Russia that are net exporters of fossil fuels, so they've spent decades lobbying to bind nuclear plant construction up in red tape to prevent them being built and funding anti-nuclear propaganda.
You can see a lot of the same attempts being made with AI, proposals to ban it or regulate it etc., or make sure only large organizations have it.
But the difference is that power plants are inherently local. If the US makes it uneconomical to build new nuclear plants, US utility customers can't easily get electricity from power plants in China. That isn't really how it works with AI.
Your point is not wrong, but I'd clarify a couple things;
The primary cost of a traditional plant is fuel. However a nuclear plant needs a tad more (qualified) oversight than a coal plant.
In the same way that the navy has specialists for running nuclear propulsion systems versus crew needed for diesel engines. Not surprisingly nuclear engines cost "more than fuel".
That cost may end up being insignificant in the long run, but cost is not zero. And shortages of staff would matter (like it does with ATC at the moment.)
Construction cost should be much lower than it is, but I don't think it'll be as cheap as say coal or diesel. The nature of the fuel would always require some extras, if only because the penalty-for-failure is so high. There's a difference between a coal-plant catastrophe and Chernobyl.
So there are costs gor running nuclear, I don't think it necessarily gets "too cheap to measure".
There are some costs that a coal plant wouldn't have, but those aren't really variable costs. If the plant's capacity is 1000MW and the grid is only demanding 500 MW right now, you don't get to send home any nuclear engineers, so what's the point in pricing kWh to discourage anyone from using more? If you build a 5GW plant instead of a 500MW plant, you don't need 10 times as many engineers just because this one's pipes have a larger diameter, so it's more economical to build larger plants, but the only times it makes sense to charge per kWh are when demand exceeds capacity and then those times become rare.
Solar is on its way to do something incredibly disruptive because it's the same "too cheap to meter" but only when the sun is shining, and then you still need the independent capacity to supply power from something else when it isn't. So now instead of "you pay ~$0.12/kWh all the time" you have a situation where power during sunshine hours is basically free but power at other times costs dramatically more than it used to because the infrastructure to supply that power has to recover its costs over significantly less usage.
> So if you build enough nuclear plants to power the grid, charging per kWh for the electricity is pointless because the marginal cost of the fuel is so low.
Wouldn't there still be the OpEx of maintaining the power plants + power infrastructure (long-distance HVDC lines, transformer substations, etc)? That isn't negligible.
...I say that, but I do note that I live in a coastal city with a temperate climate near a mountain watershed, and I literally do have unmetered water. I suppose I pay for the OpEx of the infrastructure with my taxes.
> Wouldn't there still be the OpEx of maintaining the power plants + power infrastructure (long-distance HVDC lines, transformer substations, etc)?
These are fixed costs. They don't go down if people use less power, so the sensible way to pay for them is with some kind of fixed monthly connection charge (e.g. proportional to the size of your service) or via taxes. You're still not measuring how much power people use at any given time.
Seems like the largest industrial customers would still need metered usage, though.
There are quite a few businesses that can scale limited only by power consumption — but due to that, they today need connections massively overbuilt compared to their current usage (as they project their usage growth to be extremely fast, potentially as much as doubling each year) in order to not need to be constantly re-trenching connections or browning out. Which in turn means that, under a leased-line electrical system, they'd be massively overpaying for unused power capacity at all times — possibly to the point of being unprofitable.
To achieve profitability, they'd need to negotiate billing based on only the fraction of the capacity of the available grid capacity that they're actually demanding on any given day... or, in other words, metered billing.
> Unsurprisingly, this scared the crap out of the fossil fuel industry in the US and countries like Russia that are net exporters of fossil fuels, so they've spent decades lobbying to bind nuclear plant construction up in red tape to prevent them being built and funding anti-nuclear propaganda.
This is frankly nonsense, and my hope is that this nonsense is coming from a person too young to remember the real, valid fears from disasters like 3 Mile Island, Chernobyl, and Fukushima.
Yes, I fully understand that over the long term coal plants cause many more deaths, and of course climate change is an order of magnitude worse, eventually. The issue is that human fears aren't based so much on "the long term" or "eventualities". When nuclear power plants failed, they had the unfortunate side effect of making hundreds or thousands of square miles uninhabitable for generations. The fact that societies demand heavy regulation for something that can fail so spectacularly isn't some underhanded conspiracy.
Just look at France, the country with probably the most successful wide-scale deployment of nuclear. They are rightfully proud of their nuclear industry there, but they are not a huge country (significantly smaller than Texas), and thus understand the importance of regulation to prevent any disasters. Electricity there is relatively cheap compared to the rest of Western Europe but still considerably higher than the US average.
The fears from 3 Mile Island and Fukushima were almost completely irrational. The death toll from those was too low to measure.
And the fears from Chernobyl was MOSTLY irrational.
The reason for the extreme fears that are generated from even very moderate spills from nuclear plants comes in part from the association with nuclear bombs and in part from fear of the unknown.
A lot (if not most) people shut their rational thinking off when the word "nuclear" is used, even those who SHOULD understand that a lot more people die from coal and gas plants EVERY YEAR than have died from nuclear energy throughout history.
Indeed, the safety level at Chernobyl may have been atrocious. But so was the coal industry in the USSR. Indeed, even if just considering the USSR, the death toll from coal alone caused a similar number of deaths (or a bit more) than the deaths caused by Chernobyl EVERY YEAR [1].
I think you're underselling it - the atrocious safety level at Chernobyl appears to still be an improvement on the coal industry held to a high standard. It is a horrible irony that the environmentalist movement managed to do such incredible damage to the environment by their enthusiastic attacks on nuclear.
If you want to see what a thousand square mile uninhabitable wasteland looks like, here's a YouTube video of some guys swimming in a pool underneath the Chernobyl reactor for fun: https://www.youtube.com/watch?v=WOughghZ8To
I'm so tired of hearing about how regulation is this magic salve that saves everything. Regulation is what caused Chernobyl. Soviet regulations mandated that the flawed RBMK reactor design be used. They knew it was flawed and they forced people to use it anyway. Because that's what government does. There's a similar story in western countries, where it hasn't been feasible to use better designs due to antiquated government regulations, it's just no one here has screwed up as badly as the Soviets did.
I don't buy into the "red tape" argument. For me it's all due to a lack of a business case. I think building (LWR) nuclear plants is just freaking expensive, has always been freaking expensive, was bankrolled for a while by governments because they thought it would get cheaper, and then largely abandoned in most countries when it proved to be an engineering dead end.
Here's an MIT study that dug into the reasons behind high nuclear construction cost. They found that regulatory burdens were only responsible for 30% of cost increases. Most of the cost overruns were because of needing to adapt the design of the nuclear plant to the site.
Now, you can criticize the methodology of that study, but then you have to bring your own study that shows precisely which regulatory burdens are causing these cost overruns, and which of those are in excess. Is it in excess that we have strict workplace safety regulation now? Is it in excess that we demand reactor containment vessels to prevent meltdowns from contaminating ground water supplies? In order to make a good red tape argument I expect detail in what is excess regulation, and I've never seen that.
Besides, if "red tape" and fossil industry marketing was really the cause of unpopularity, and the business case for nuclear was real when installing plants at scale, you would see Russia and China have the majority of their electricity production from nuclear power.
- Russia is the most pro-nuclear country in the world, and even they didn't get past 20% of electricity share. They claim levelized cost for nuclear on the order of that of solar and wind, but I am very skeptical of that number, and anyone who knows anything about the Russian government's relation to the truth will understand why. When they build nuclear plants in other countries (e.g. the bangladesh project) they are not that cheap.
- China sits at low single digit percentages of nuclear share, with a levelized cost that is significantly higher than Russia's and well above that of solar and wind. While they're planning to grow the nuclear share they assume it will be based on new nuclear technology that changes the business case.
Both Russia and China can rely on cheap skilled labor to bring down costs, a luxury western countries do not have.
And this is ultimately the issue: the nuclear industry has been promising designs that bring down costs for over half a century, and they have never delivered on that promise. It's all smoke and mirrors distracting from the fact that building nuclear plants is inherently freaking expensive. Maybe AI can help us design better and cheaper nuclear power plants, but as of today there is no proven nuclear plant design that is economical to build, and that is ultimately why you see so little new nuclear plant construction in the west.
It is worth noting that the effect is jaw-droppingly stark. The regulators managed to invert the learning curve [0] so the more power plants get built the more expensive it gets! It is one of the most stunning failures of an industrial society in the modern era; the damage this did to us all is huge. It is disheartening that our leadership/society chose to turn their backs on the future and we're all lucky that the Chinese chose a different tack to the West's policy of energy failure. At least there are still people who believe in industry.
> The regulators managed to invert the learning curve [0]
This is conjecture. If you wanted to establish this, you would have to show that cost of (skilled) labor was unchanging or negligible.
It is also important to consider that nuclear power deaths/damages are much more localized and traceable than excess deaths from air pollution, and thus much less acceptable to the voting population-- you could argue that this should not make any difference (I disagree), but I don't want to digress here too much.
> we're all lucky that the Chinese chose a different tack to the West's policy of energy failure
What do you believe that is? Because from my point of view, China generates a negligible amount of electricity from nuclear power (<5%), this is not going to change within the next decades, and the main "purpose" from what I can tell is to in-house reactor/turbine know-how (instead of relying on Alstom/Siemesn).
Those are constant dollars. Are you claiming that the real cost of labour went up 4- to 8-fold in the nuclear industry? Why did that happen? The median nuclear plant construction worker would be making $240k/annum type wages. And as I recall I've not heard of that sort of wage rise outside a regulatory failure or somewhere like China undergoing a massive economic boom.
> It is also important to consider that nuclear power deaths/damages...
Maybe you can answer this for me - what deaths and damages? So far I've never been able to pin down any actual death or damage to a nuclear meltdown. I'm sure there are some, but most of the actual attempts to quantify it require appealing to hypothetical deaths and damages that no-one can specifically point to, or tiny numbers that are irrelevant to industrial policy.
I know people who lived in a town next to a lead-zinc mine. That appears to be about as bad as a nuclear crisis from what I can gather and it doesn't seem to be causing anyone undue stress. We're still using lead and zinc. People still live in the town.
Direct death counts from nuclear meltdowns might be rather low, but the damage is clearly that entire cities need to be evacuated (and stay uninhabitable for decades). Fukushima alone cost the Japanese taxpayer close to $200 billion.
> Those are constant dollars. Are you claiming that the real cost of labour went up 4- to 8-fold in the nuclear industry?
I'm not saying that is the only effect. But if you want to blame "onerous regulations" for nuclear power being so expensive, then you have to at least put bounds on other factors driving cost (and you also have to show those relaxed regulations would not have led to a significant amount of additional incidents; the current rate of 2 meltdowns for ~500ish reactor sites is already problematic enough).
I personally think that collective standards regarding risk and pollution have risen significantly since the early days of nuclear power, and a lot of things that were done back then would be considered unacceptably reckless/negligient nowadays (=> Hanford became a superfund site for a reason), so claiming that those historical costs are repeatable seems very dubious in the first place to me.
> Direct death counts from nuclear meltdowns might be rather low, but the damage is clearly that entire cities need to be evacuated (and stay uninhabitable for decades). Fukushima alone cost the Japanese taxpayer close to $200 billion.
Ok, so say they don't evacuate the city. What is the actual risk here that we're talking about?
> I'm not saying that is the only effect.
What other effect are you considering? The materials aren't getting that more expensive and the labour will cost similar too. There aren't a lot of options left apart from regulatory changes.
> I personally think that collective standards regarding risk and pollution have risen significantly...
Then do you potentially think that the reason the price is rising is because of regulation? Because this whole paragraph reads like a justification for the costs incurred due to regulation. I'm not seeing what your complaint is here pinning the inverted learning curve on the regulators - you seem to be saying that is what happened and it is reasonable in this paragraph.
> Ok, so say they don't evacuate the city. What is the actual risk here that we're talking about?
The risk is that hundreds of people avoidably die of cancer over the following decades, and that you lose an immense amount of trust and credibility in government. This is not easy to de-risk because there are a lot of potential mechanisms for bio-accumulation/concentration, and by the time you find out what those exact mechanisms are empirically, the damage might already be done (and by then its too late, and people are no longer gonna trust anything you tell them).
I'm not arguing that the Fukushima evacuation was 100% the right call to make even in hindsight, but it was most certainly a defensible position (even in retrospect).
> Then do you potentially think that the reason the price is rising is because of regulation?
Strongly disagree with the framing. My opinion is that most early reactors did not meet modern safety/pollution expectations and quite frequently resulted in a toxic mess that still requires cleanup today (Sellafield would be a non-US example).
Despite all regulations, the safety track record (2 meltdowns at 500ish reactor sites) is arguably really poor, especially when reactor meltdowns had been portrayed as "virtually impossible" by early proponents.
> What other effect are you considering?
The very same effects that make bespoke heavy machinery comparatively expensive nowadays in general:
- Low benefit from industrial automation
- Mechanical engineering sectors have shrunk in the west (=> parts/subassemblies and qualified labor more expensive)
- Generally higher wages
Those apply much less to renewables because those are less labor intensive and benefit from economies of scale more (by design). Its also much easier to benefit from cheap foreign labor with those (especially solar).
As a sanity check:
I would always expect coal power plants to stay significantly cheaper than nuclear power (when disregarding CO2/fuel cost). But even in that setting, modern coal power (just like nuclear power) already struggles to compete with solar/wind on levelized cost basis today (and trends are very clear!)
So even if we dropped all nuclear regulation today completely (with disastrous likely effects) then the absolute best we could hope for would be to get competitive (on levelized cost) with the most expensive forms of renewables (offshore wind and rooftop solar). Given that it would likely take decades to act on such a plan, all the risks (and current downward trends on renewables and storage costs) chasing that approach seems utterly pointless to me.
Thus I believe the current nuclear approach (build a few reactors here and there so the know-how is not lost completely), which is exactly what both China and the US are doing, is perfectly sensible.
> The risk is that hundreds of people avoidably die of cancer over the following decades
For long-term displacement, many people (mostly sick and elderly) died at an increased rate while in temporary housing and shelters. Degraded living conditions and separation from support networks are likely contributing factors. As of 27 February 2017, the Fukushima prefecture government counted 2,129 "disaster-related deaths" in the prefecture. This value exceeds the number that have died in Fukushima prefecture directly from the earthquake and tsunami. .... As of the year 2016, among those deaths, 1,368 have been listed as "related to the nuclear power plant" according to media analysis
Do you actually mean 100s over decades? Given that the evacuations from Fukushima caused thousands (well, thousand - but we're talking the highest hundreds before we get to thousands) of deaths in years, it seems like the risk of nuclear meltdowns would be completely acceptable. The government forced people to accept higher risk because bureaucrats panicked and it seems that was acceptable. What is the problem with a meltdown supposed to be if more damage is done just for theatre?
You're proposing people respond to a threat that isn't really there. Hundreds of people routinely die over decades from all sorts of industrial processes. A once in a generation risk that somewhere on the globe will suffer, maybe, 100s of deaths over the course of a few decades is nothing. Coal is still worse, is everywhere and people accept that. Is this the limit of the damage you're worried about?
So do coal plants. Interestingly approximately nobody seems to be worried about the thousands of coal ash ponds leaching radioactive material, toxic heavy metals and teratogenic compounds into ground water on a regular basis - while storing spent fuel element in a salt mine thousands of miles away engenders outrage…
I have never worried about a coal plant in Ukraine. I lived at a place that was potentially downwind of Chernobyl and inside the projected bad-things-happening plume. The scale of a nuclear disaster well exceeds the scale of a coal plant disaster.
Many things are cheap when you ignore externalities.
The thing is, nuclear was never on such a steep learning curve as solar and batteries are today.
It’ll never be too cheap to meter, but electricity will get much cheaper over the coming decades, and so will synthetic hydrocarbons on the back of it.
Your electricity bill is set by the grift of archaic fossil energy industries. And nuclear qualifies as a fossil industry because it's still essentially digging ancient stuff out of the ground, moving it around the world, and burning it in huge dirty machines constructed at vast expense.
There are better options, and at scale they're literally capable of producing electricity that literally is too cheap to meter.
The reasons they haven't been built at scale are purely political.
Today's AI is computing's equivalent of nuclear energy - clumsy, centralised, crude, industrial, extractive, and massively overhyped and overpriced.
Real AI would be the step after that - distributed, decentralised, reliable, collaborative, free in all senses of the word.
watch the cost of electricity go up because the demand created by data centers. i'm building an off grid solar system right now and it ain't cheap! thinking about a future where consumers are competing with data centers for electricity makes me think it might feel cheap in the future, though.
bro, how much did your electricity cost go up because of millions of people playing games on their 500w+ gpus? by a billion of people watching youtube? by hundreds of millions of women and children scrolling instagram and tiktok 8 hours a day?
Well, Altman is also investing in Helion, which projects to get the price of electricity to ~$10/MWh, but for whom, much like solar, wind, and actual nuclear the cost structure is overwhelmingly dominated by capital costs and non-varying capital costs (the cost of uranium or Helion's fuel will be negligible vs capital and manpower). So there's actually a pretty good reason to think long term electricity will be marginally so cheap that it isn't metered but instead basically bought in chunks of capacity or availability.
Another way for intelligence to get too cheap to meter is for the cost to fall so low it becomes hyperabundant. If you were to, for instance, take AI2027 as a benchmark and think ultimately we'll achieve something like the equivalent of John von Neumann in a box with a 2T dense equivalent parameter model and it will match such a Nobel prize winner's productivity when running inference at say 15 tokens a second (as fast as people can read) then you only need in principle 60 teraflops of AI infernce compute, which is roughly 2x the current Apple Neural Engine. So plausibly by the time you get to the 2030s, every laptop, smartphone, etc will be easily able to run models as powerful as the smartest people.
Somewhat longer term, I'm sure Altman expects the entire process to be automated and for the computational efficiency to rise significantly. If you take recent estimates from various players in the reversible computing space, you'd guesstimate that you ought to be able to do 60tflops by the late 2030s using under 0.1W or ~1kWh/yr which Helion could produce for ~1¢. I do feel like 1 year of cognitive labor from the smartest person a penny or two renders intelligence too cheap to meter out on a per-hour basis.
Right before I came to this article, I watched a video about Alpha fold determining the structure of almost all proteins, and the mention of other programs working on other forms of crystal structures( Magnets, superconductors) I am staring to feel the acceleration. I wish I felt more hopeful.
> We are past the event horizon; the takeoff has started. Humanity is close to building digital superintelligence, and at least so far it’s much less weird than it seems like it should be.
Not sure if wishful thinking trying to LARP-manifest this future into being, or just more unfalsifiable thinking where we can always be said to be past the event horizon and near to the singularity, given sufficiently underwhelming definitions of "event horizon" and "nearness."
Agreed. I realize it’s a metaphor but saying we’re so close to the singularity that it’s guaranteed to happen (the event horizon being a point of no return) seems like wild hubris.
Especially considering the global instability we're facing right now. The World Bank released new estimates US GDP growth in 2025, falling from roughly 2.5 to 1.5 percentage points. I don't think most people would predict the singularity beginning with less economic growth than we saw in the 80s
Of course, progress could stall out, but we appear to have sufficient compute to do anything a human brain can do, and in areas, AIs are already far better than humans. With the amount of brain power working in, and capital pouring into, this area, including work on improving algorithms, I think this essay is fundamentally correct that the takeoff has started.
> in areas, AIs are already far better than humans.
You could say the same thing about a CPU from 40 years ago - they can do math far better than humans. The problem is that there are some very simple problems that LLMs can’t seem to reliably solve that a child easily could, and this shows that there likely isn’t actual intelligence going on under the hood.
I think LLMs are more like human simulators rather than actual intelligent agents. In other words, they can’t know or extrapolate more knowledge than the source material gives them, meaning they could never become more intelligent than humans. They’re like a very efficient search engine of existing human knowledge.
Can anyone give me an example of any true breakthrough that was generated by an LLM?
They could become better at humans bc there's a RL training loop? I don't understand how this isn't directly clear. Even raining purely on human data it's possible to be mildly superhuman (see experiments of chess AI trained on human data only) but once you have verifiable tasks and RL loop the human data is just Kickstarter
You might be correct about LLMs. Let's say that you are.
40 years ago we were clearly compute bound. Today, I think it's fairly clear we are not; if there is anything a human can do that an AI can't, it's because we lack the algorithms, not the compute.
So the question becomes, now that we have sufficient compute capacity, how long do you think it will take the army of intelligent creative humans (comp sci PhDs, and now accelerate by AI assistance) to develop the algorithmic improvements to take AI from LLMs to something human level?
Nobody knows the answer to the above, and I could be very wrong, but my money would bet on it being <30 years, if not dramatically sooner (my money is on under 10).
It seems to me like the building blocks are all here. Computers can now see, process scenes in real time, move through the world as robots, speak and converse with humans in real time, use tools, create images (imagine?), and so forth. Work is continuing to give LLMs memory, expanded context, and other improvements. As those areas all get improved on, tied together, recursively improved, etc., at some point I think it will be hard to argue it is not intelligence.
Where we are with LLMs is Kitty Hawk. The world now knows that flight (true human level intelligence) is possible and within reach, and I strongly believe the progress from here on out will continue to be rapid and extreme.
> So the question becomes, now that we have sufficient compute capacity, how long do you think it will take the army of intelligent creative humans (comp sci PhDs, and now accelerate by AI assistance) to develop the algorithmic improvements to take AI from LLMs to something human level?
This assumes that the eventual breakthroughs start from something like LLMs. It's just as likely or more that LLMs are an evolutionary dead end or wrong turn, and whatever leads to AGI is completely unrelated. I agree that we are no longer compute bound, but that doesn't say anything about any of the other requirements.
I get the feeling the first paragraph was intended to make people who immediately went "this is is absolute horseshit" stop reading, in the same way an obvious spam email does
556 comments
[ 3.6 ms ] story [ 262 ms ] threadSo when AGI comes, I am curious what the new jobs are?
I see that prompt engineer is one of the jobs created because it's the way to ask a LLM certain tasks, but now AI can do this too.
I'm thinking that any new jobs AI would make, AI would just take them anyway.
Are there new jobs coming from this abundance that is on the horizon?
There's no reason to be confident that such a future will arrive without difficult moral questions, or that it's as simple as a #define FREE_WILL 0 .
Yes: we still have a long way to go in restoring equilibrium to the climate and producing more sustainable alternatives to our current cities, products and materials. It's a megaproject which could take hundreds of years, and will require plenty of human involvement.
The planet is going to be fine either way, but if capitalism doesn't figure out how to price in externalities, it will slowly run out of human consumers.
All that I hope for in this case is that governments actually take this seriously and labs/governments/people work together to create better societal systems to handle that. Because as it stands, under capitalism I don't think anyone is going to willingly give up the wealth they made from AI to spread to the populus as UBI. This is necessary in some capitalist system (if we want to maintain that) since its built on consumption and spending.
Though if its truly an "abundance" scenario then I'd imagine it probably wouldn't matter that people don't have jobs since I'd assume everything would be dirt cheap and quality of life would be very high. Though personally I am very cynical when it comes to "agi is magic pixie dust that can solve any problem" takes, and I'd assume in the short term companies will lay off people in swathes since "AI can do your job," but AI will be nowhere close to increasing those laid-off people's quality of life. It'll be a tough few years if we don't actually get transformative AI.
> 2026 will likely see the arrival of systems that can figure out novel insights
Interesting the level of confidence compared to recent comments by Sundar [1]. Satya [2] also is a bit more reserved in his optimism.
[1] https://www.windowscentral.com/software-apps/google-ceo-agi-...
[2] https://www.tomshardware.com/tech-industry/artificial-intell...
well yeah, you can't continue the ponzi scheme if you say "aw shit, it's gonna take another 10 years"
Microsoft/Google will continue to exist without, OpenAI won't
Here he says:
> Intelligence too cheap to meter is well within grasp.
Six months ago[0] he said:
> We are now confident we know how to build AGI as we have traditionally understood it.
This time:
> we have recently built systems that are smarter than people in many ways
My summary: ChatGPT is already pretty great and we can make it cheaper and that will help humanity because...etc
Which moves the goal posts quite a bit vs: we'll have AGI pretty soon.
Could be he didn't reiterate we'd have AGI soon because he thought that was obvious/off-topic. Or it could be that he's feeling less bullish, too.
[0]: <https://blog.samaltman.com/reflections>
Does anyone know if there are well established scaling laws for reasoning models similar to chinchilla scaling. (i.e. is the above claim valid?)
I heard similar things in my college dorm, amid all the hazy smoke.
It’s very difficult to take this stuff seriously. It’s like the initial hype around self driving cars wound up by 1000x. Because we got from 1 to 100 of course we’ll get from 100 to 200 in the same amount of time. Or less! Why would you even question it?
I would not be opposed to living in a future where I can personally live in space. It would be quite fun.
[1] (paywalled) https://fortune.com/2025/06/06/google-deepmind-ceo-demis-has...
Unless people are envisioning living in magical holodecks all the time, with magical food replicators? But those don't come along automatically with "space", no matter how much Star Trek you've seen...
(my comparison is incomplete though, it doesn’t factor in that these two also have a huge financial incentive to be hyping this stuff up)
What is that man smoking and can I have some?
Five years wouldn’t be enough time to “colonise” Antarctica, let alone another planet (just one!), and certainly not anything at a larger scale, even if we were visited by aliens tomorrow and they gifted us five hundred spaceships to give us a boost.
> “If that all happens, then it should be an era of maximum human flourishing, where we travel to the stars and colonize the galaxy. I think that will begin to happen in 2030.”
You are confusing "era of maximum human flourishing ... begin to happen" with "have colonized galaxy".
It turned out automating creative output, like art or writing, at least to be competitive enough with entry-level humans, was much easier, and the consequences for getting bad output are much less serious, and far short of death, seriously injury, and major financial liability as with self-driving. Fields like concept art and copy editing are already being devastated by generative AI. Voice overs too.
And looking at Google's Veo, I can easily see this tech being used to generate short ads, especially for YouTube, where before you would have had to hire human actors, cameramen, sound/lighting people, and an editor.
Wait... Self driving cars actually happened. Seems like the hype was true, but 5 years later than expected and now people just think it's normal so they feel the hype was overblown.
I've always been a proponent that we would see self driving cars in our lifetime. But they have absolutely not arrived outside of the sheltered enclaves of a handful of tech-centered cities.
In my opinion, it's as similarly off-base as to making claims that 'mach speed consumer air travel' "arrived" just because for a few decades you could catch a Concorde flight between NYC and London.
You’re right that they have a long way to go before being fully mainstream. But thousands of people are using them every day in multiple major cities. That’s a pretty big milestone.
Not able to handle what the streets of Los Angeles throw at it, however.
None of this is even close to happening. Waymo is impressive tech, for sure, but what they're proving is that this is just not currently solvable as a general problem. The best we can do is to meticulously craft a solution for one constrained use case - driving in SF, driving in LA, etc. And even then, we need to pick certain use cases - it's not like they could use their current tech to start autonomous service in Juneau Alaska. Or even NYC, most likely.
[0] https://news.ycombinator.com/item?id=23886325
I'll believe self-driving is solved when I can order an automated ride to a BLM/forest service dirt road I only have GPS coordinates for and that has cell service on the hilltop only.
Might as well move it to "as long as there's no semi crossing the road" to cover for Tesla.
Centrally-managed clean and rigid setting is not the full extent of the problem domain.
As I said, I'll believe it's solved when these limitations are no longer necessary for it to work. You can choose to believe it based on a research project or marketing materials.
For most people, they'll consider self-driving as "happened" when most cars are self-driving. Similarly, the consider the smartphone as "happened" not when the palm came out - but somewhere around the iPhone 4.
Like, right now, immunotherapy for cancer has "happened" - there's real patients really doing it, for real. But most cancer patients don't consider immunotherapy as "happened", they're still getting chemo. Once chemo is obsolesced, we can consider immunotherapy as "happened". From then, to now, may be on the scale of decades.
This isn't correct: people want good software and good art, and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the hallucination issues of LLMs.
> As datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity. (People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours
This is the first time a number has been given in terms of ChatGPT's cost-per-query and is obviously much lower than the 3 watts still cited by detractors, but there's a lot of asterisks in how that number might be calculated (is watt-hours the right unit of measurement here?).
Most art is just meh and earns billions.
I don’t see AI stealing art. Most modern arts have a huge social component.
https://www.youtube.com/watch?v=k85mRPqvMbE
One aspect about modern generative AI that has surprised me is that it should have resulted in a new age of creatively surreal shitposts, but instead we have people Ghiblifying themselves and pictures of Trump eating tacos.
https://www.instagram.com/reel/DE3EhrZyCVd/
That's how electricity is most commonly priced. What else would you want?
I still find the Technology Connections video counterintuitive. https://www.youtube.com/watch?v=OOK5xkFijPc
A watt needs a time unit to be converted to joules because a watt is a measure of power, while a joule is a measure of energy.
Think of it like speed and distance:
A watt is a RATE of energy use, just like speed (e.g., miles per hour) is a RATE of travel.
A joule is a total amount of energy, just like distance (e.g., miles) is a total amount of travel.
If you're traveling 60 mph but you don't specify for how long... You won't know how far you went, how much gas you used, etc.
There are situations where Watts per se matter, eg if you build a datacenters in an area that doesn't have enough spare electricity capacity to generate enough watts to cover you, and you'd end up browning out the grid. Or you have a faulty burner and can't pump enough watts into your water to boil it before the heat leaks out, no matter how long you run it.
what he has left unsaid, is that electricity demand will rise substantially
good luck heating your home when you're competing with "superintelligence" for energy
[0] To my knowledge, no desktops are built to max out their power supply for an extended time; this is a simplification for illustration purposes.
At some point you’ll need to measure in terms of $ earned per W-H, or just $out/$in.
Money is irrelevant to energy efficiency.
My point was twofold, “the average query” becomes less meaningful as the variance increases. Sure one can _in principle_ report W-h spent on your account or query, but I think it will get more opaque and hard to predict. Average becomes less useful when agents can do an unpredictable amount of work with increasingly large bounds.
Second, and this was perhaps worthy of expanding in my post - I think in the radical abundance world that Altman is describing, energy and efficiency stops being something that people talk about. Fusion, space based solar farms, whatever, it’s easy to imagine solving this stuff. And I think you can even imagine this happening sooner if for example one provider stamps a “100% renewable datacenter” option on their AI product. Then you might not care about energy efficiency at all; in which case you just care about profit.
You are getting confused here...
LLMs are terrible at doing software engineering jobs. They fall on their face trying to work in sprawling codebases.
However, they are excellent at writing small bespoke programs on behalf of regular people.
Becky doesn't need ChatGPT to one-shot Excel.exe to keep track of the things she sells at the craft fair that afternoon.
This is a huge blind-spot I keep seeing people miss. LLM "programmers" are way way more useful to regular people than they are to career programmers.
This smells like "Self-checkouts", "perfect bespoke solution so that everyone can check their own groceries out, no need to wait for a slow/expensive _human_ to get the job done, except you end up basically just doing the cashiers job and waiting around inconveniently whenever you want to get some beer
I'm also pretty anti-social and actually prefer the robotic banter of quickly checking out with a person to the anxious nightmare of just trying to buy something real quick and now waiting because the machine hates life more than me.
Oh and many of the folks doing the bagging at some of the stores are disabled, and I dunno - I hope we're taking care of people in those jobs.
Instead of a guard there's a scanner by the exit gate where you scan your ticket. In my case just a small stub since the actual ticket is sent digitally.
I think it works so smoothly, much better than before this system was introduced. My recent experience buying groceries in Belgium was very similar.
And the scanner to exit is terrible for fire safety and accessibility. Also I've seen it break with the result that you had to trigger the alarm to leave.
It does not work smoothly at all, but perhaps you live in a different sweden than I.
I use this method of shopping whenever available.
And for items like bread or so you need to navigate a number of menus to find it.
And of course you are asked if you are a member, if you'd like to become a member, if you want to do a donation, and a number of other useless questions.
Yes you do need to be a member to use the system, but that's a very small price to pay for the convenience and speed of avoiding all lines and stress of the regular checkout.
Going from "a supermarket near my home" to "every single supermarket in sweden" is kinda of a big leap.
Try ICA in Olskroken or Godhemsgatan for example…
I Most supermarkets have self checkout, but even that isn't omnipresent (and it's usually slower).
The other problem is that, like self checkout, it often requires human intervention (buying alcohol or high value items, for examples) still requires human intervention. This can require a long wait at the end. I once got so sick of waiting that I left my shopping and walked away and went somewhere else.
That's exactly why she asks an LLM to do it for her. A program like this would be <1k LOC, well within even the current LLM one-shot-working-app domain.
The lower barrier to entry might mean the average quality of software goes WAY down. But this could be great if it means that there are 10x as many little apps that make one person happy by doing only as much as they need.
The above is also consistent with the quantity of excellent software going way up as well (just slower than the below-average stuff).
Fortunately AI helps with that too.
It doesn’t give you full control, but I believe it gives you enough control to do what I proposed for most consumers.
We have these really amazing computers with very high-quality software but still, so many processes are manual because we have too many hops from tool to tool. And each hop is a marginal increase in complexity, and an increase in risk of things going wrong.
I don't think AI will solve this necessarily, and it could make it worse. But, if we did want to improve it, AI can act as a sort of spongy API. It can make decisions in the face of uncertainty.
I imagine a world where I want to make a doctor's appointment and I don't open any apps at all, or pick up the phone. I tell my AI assistant, and it works it out, and I have all my records in one place. Maybe the AI assistant works over a phone line, like some sort of prehistoric amalgamation of old technology and new. Maybe the AI assistant doesn't even talk to a receptionist, but another AI assistant. Maybe it gives me spongy human information, but speaks to other's of it's kind in more defined interfaces and dialects. Maybe, in the future, there are no applications at all. Applications are, after all, a human construction. A device intended to replace labor, but a gross approximation of it.
>However, they are excellent at writing small bespoke programs
For programmers I predict a future of a million micro services.
Sprawling has always been an undesirable and unnecessary byproduct of growing code bases, but there's been no easy solution to that. I wonder if LLMs would perform better on a mono repo of many small services than one sprawling monolith.
It still happens but it's not a favorite experience anymore. It's just a source of loathing for MBA culture.
https://en.wikipedia.org/wiki/Apparatchik
Been saying that for years. Private equity is perhaps analogous to the Politburo.
Cloud providers make more money the more they can get people to use inefficient designs with more moving parts to rack up more charges. Bonus if it also locks you into managed services. Double bonus if those are proprietary. Complexity benefits cloud hosts.
Microservices, like all patterns, sometimes make sense. Like all patterns they often get overused.
But end users often only need a tiny fraction of what the software is fully capable of, leaving a situation where you need you need to purchase 100% to use 5%.
LLMs can break down this wall by offering the user the ability to just make a bespoke 5% utility. I personally have had enormous success doing this.
Sure, an LLM might be able to guide you through the steps, and even help when you stumble, but you still have to follow a hundred little steps exactly with no intuition whether things will work out at the end. I very much doubt that this is something many people will even think to ask for, and then follow-through with.
Especially since the code quality of LLMs is nowhere near what you make it out to be. You'll still need to bear with them and go through a lot of trial and error to get anything running, especially if you have no knowledge of the terms of art, nor any clue of what might be going wrong if it goes wrong. If you've ever seen consumer bug reports, you might have some idea of the quality of feedback the LLM may be getting back if something is not working perfectly the first time. It's very likely to be closer to "the button you added is not working" than to "when I click the button, the application crashes" or "when I click the button, the application freezes for a few seconds and then pops up an error saying that input validation failed" or "the button is not showing up on the screen" [...]
I’m radiologist, I’ve been paying for software that sped up my reporting like 200 usd per month. I’ve remade all the functionality I need in one evening with cursor and added some things that I’ve found missing from the original software.
So far I have created 7 programs that are now used daily in production. One of them replaces a $1k/yr/usr CAD package, and another we used to bring in a contractor to write. The others a miscellaneous apps for automating/simplifying our in houses processes. None of the programs are more than 6k LOC.
Kinda. They're good, and I like them, but I think of them like a power tool: just because you can buy an angle grinder or a plasma cutter from a discount supermarket, doesn't mean the tool is safe in the hands of an untrained amateur.
Someone using it to help fix up a spreadsheet? Probably fine. But you should at least be able to read the code to the level you don't get this deliberate bad example to illustrate the point:
Still useful, still sufficiently advanced technology that for most people it is (Clarketech) magic, but also still has some rough edges.Absolutely! We’re in an interesting time, and LLMs are certainly over-hyped.
With that said, I’d argue that most software today is _average_. Not necessarily bad by design, but average because of the (historic) economies of software development.
We’ve all seen it: some beloved software tool that fills a niche. They raise funding/start to scale and all of a sudden they need to make this software work for more people.
The developers remove poweruser features, add functionality to make it easier to use, and all of a sudden the product is a shell of its former self.
This is what excites me about LLMs. Now we can build software for 10s or 100s of users, instead of only building software with the goal (and financial stability) of a billion users.
My hope is that, even with tons of terrible vibe coded atrocities littering the App Store/web, we’ll at least start to see some software that was better than before, because the developers can focus on the forest rather than each individual tree (similar to Assembly -> C -> Python).
:-)
The body uses 25w resting and thus the brain is about 5w.
Source: biology degree but like I said please take with the same amount of weight as a hallucinating LLM.
Resting energy usage in humans is ~1200–1500 kcal/day, or about 60–70 watts, depending on the person. Logic holds, estimate is just low
Sure, we'll all get a subscription and subject ourselves to biometric verification in order to continue our profession.
OpenAI's website looks pretty bad. Shouldn't it be the best website in existence if the new Übermensch programmers rely on "AI"?
What about the horrible code quality of "AI" applications in the "scientific" Python sphere? Shouldn't the code have been improved by "AI"?
So many questions and no one shows us the code.
I used to believe this wasn't worth considering (because while training is energy-intensive, once the model is trained we can use it forever.) But in practice it seems like we switch to newer/better models every few months, so a model has a limited period of usefulness.
> (is watt-hours the right unit of measurement here?)
Yes, because we want to measure energy consumed, while watts is only a measure of instantaneous power.
Other people know a novel cannot be written by a machine— because a novel is human by definition, and AI is not human, by definition.
It’s like wondering if a machine can express heartfelt condolences. Certainly a machine can string words together that humans associate with other humans who express heartfelt condolences, but when AI does this they are NOT condolences. Plagiarized emotional phrases do not indicate the presence of feeling, just the presence of bullshit.
I thought a novel was just a decent length fiction book?
The purpose of fiction at least isn't to express true feeling, it's to instill feeling in the reader. To that end it doesn't matter the process that produces the words.
Is AI going to win the Pulitzer any time soon? Nah.
Is AI going to be able to produce novel length works of fiction that are good enough for at least some people in the near future? Most likely.
Anything is good enough if your standards are low.
How about affordable housing?
At the very least, moving heavy extractive industries out of the biosphere would be a sensible allocation of resources: metal and energy are common in the solar system, but life supporting ecosystems are not.
(Or both)
The average person on earth would love it we could have readily available clean water, cheaper housing and food, reliable healthcare to retire without worrying.
That would actually be blissful beyond our wildest dreams: everyone around you and beyond being able to have a peaceful life.
Then, I just buy houses in High COL areas and start popping them up and moving families in ASAP. What are they gonna do evict working families? Good luck
Then dissolve the underlying corporation before the municipal fines kick in and escape into the night. Render municipal and neighborhood control laws meaningless.
*may not be humanoid, I'm skirting that debate here.
This seems optimistic.
It's simply pitting the most dysfunctional municipal and judicial systems against each other with the default outcome being people being housed.
The municipality can’t take away the land (most of the value here) and putting the house on top will be relatively cheap, given the low total labor costs.
If the whole affair is revenue neutral, that's a huge win.
they do this all the time!
But space colonization is cool and poor people are yickie.
It was the one genuine, serious potential improvement.
But technology often can solve political problems by shifting the balance of power. Come up with more ways to make it easier for people to live away from high cost of living areas, for example, so that governments with high housing costs start losing tax base to other jurisdictions.
Between this and Ed Zitron at the other end of the spectrum, Ed's a lot more believeable to be honest.
"probably", "if they embrace the new tools". Hard to read anything but contempt for the role of humans in creative endeavors here, he advocates for quantity as a measure of success.
Gary has invested heavily in an anti-AI persona, continually forecasting AI Winters despite wave after wave of breakthroughs.
Sam, on the other hand, is not just an AI enthusiast; he speaks in a manner designed to build the brand, influence policy, and continuously boost OpenAI's valuation and consolidate its power. It's akin to asking the Pope whether Catholicism is true.
Of course, there might indeed be significant roadblocks ahead. It’s also possible that OpenAI might outpace its competitors—although, as of now, Gemini 2.5 Pro holds the lead. Nevertheless, whenever we listen to highly biased figures, we should always take their claims with a grain of salt.
That’s heresy, with this crowd. Everyone wants to be The Gatekeeper, and get filthy rich.
it's super competitive, which should be good for innovation, but there's also significant incentive to use PR tactics to sell that innovation for much more than it's worth.
Sam's comments about how we're super close to AGI fall flatter-than-ever, after the latest model releases (from all players) and the Apple paper confirming what everybody already knew.
If you're thinking about the apple paper, you should know that its methodology was flawed in many ways, and their findings absolutely do not support the catchy title. But a lot of slop was generated because negativity + catchy title + apple = hits.
https://www.wheresyoured.at/openai-is-a-systemic-risk-to-the...
I see a similar thing at work — the number of projects a developer can get through isn’t bounded by the lines of code they can churn out in a day. Instead it’s bounded by their appetite for getting shouted at when something goes wrong.
Until you can shout at an LLM, I don’t think they’re going to replace humans in the workplace.
Viewed through that lens, LLMs and their total lack of accountability are a perfect match for the modern-day business; that's probably part of why so many executives are hell-bent on putting them everywhere as quickly as possible.
I think you answered this with the rest of your comment.
It was not obvious to me that this was the author’s intent.
Our epistemological standards are so shitty and so pathetically poor, that it's hardly surprising that we're fooled by LLMs much as we are by academic bullshit artists.
We're obviously not quite there yet, but I don't see any inherent limitation to human managers' ability to shout at AI agents. I'll go even a step further and say that knowing that the AI doesn't actually have feelings that I can affect, and that it has limited context helps make my shouting more productive, focusing on the task at hand, rather than on its general faults, as I might when I'm angry at a human.
"Hey, it'd be a shame if somethin', uh, happened to that nice bit of expertise ya got there, y'know. A darn shame."
Famous last words.
>Intelligence too cheap to meter is well within grasp
And also:
>cost of intelligence should eventually converge to near the cost of electricity.
Which is a meter-worthy resource. So intelligence effect on people's lives is in the order of magnitude of one second of a toaster use each day, in present value. This begs the question: what could you do with a toaster-second say 5 years from today?
Unsurprisingly, this scared the crap out of the fossil fuel industry in the US and countries like Russia that are net exporters of fossil fuels, so they've spent decades lobbying to bind nuclear plant construction up in red tape to prevent them being built and funding anti-nuclear propaganda.
You can see a lot of the same attempts being made with AI, proposals to ban it or regulate it etc., or make sure only large organizations have it.
But the difference is that power plants are inherently local. If the US makes it uneconomical to build new nuclear plants, US utility customers can't easily get electricity from power plants in China. That isn't really how it works with AI.
The primary cost of a traditional plant is fuel. However a nuclear plant needs a tad more (qualified) oversight than a coal plant.
In the same way that the navy has specialists for running nuclear propulsion systems versus crew needed for diesel engines. Not surprisingly nuclear engines cost "more than fuel".
That cost may end up being insignificant in the long run, but cost is not zero. And shortages of staff would matter (like it does with ATC at the moment.)
Construction cost should be much lower than it is, but I don't think it'll be as cheap as say coal or diesel. The nature of the fuel would always require some extras, if only because the penalty-for-failure is so high. There's a difference between a coal-plant catastrophe and Chernobyl.
So there are costs gor running nuclear, I don't think it necessarily gets "too cheap to measure".
Solar is on its way to do something incredibly disruptive because it's the same "too cheap to meter" but only when the sun is shining, and then you still need the independent capacity to supply power from something else when it isn't. So now instead of "you pay ~$0.12/kWh all the time" you have a situation where power during sunshine hours is basically free but power at other times costs dramatically more than it used to because the infrastructure to supply that power has to recover its costs over significantly less usage.
Wouldn't there still be the OpEx of maintaining the power plants + power infrastructure (long-distance HVDC lines, transformer substations, etc)? That isn't negligible.
...I say that, but I do note that I live in a coastal city with a temperate climate near a mountain watershed, and I literally do have unmetered water. I suppose I pay for the OpEx of the infrastructure with my taxes.
These are fixed costs. They don't go down if people use less power, so the sensible way to pay for them is with some kind of fixed monthly connection charge (e.g. proportional to the size of your service) or via taxes. You're still not measuring how much power people use at any given time.
There are quite a few businesses that can scale limited only by power consumption — but due to that, they today need connections massively overbuilt compared to their current usage (as they project their usage growth to be extremely fast, potentially as much as doubling each year) in order to not need to be constantly re-trenching connections or browning out. Which in turn means that, under a leased-line electrical system, they'd be massively overpaying for unused power capacity at all times — possibly to the point of being unprofitable.
To achieve profitability, they'd need to negotiate billing based on only the fraction of the capacity of the available grid capacity that they're actually demanding on any given day... or, in other words, metered billing.
This is frankly nonsense, and my hope is that this nonsense is coming from a person too young to remember the real, valid fears from disasters like 3 Mile Island, Chernobyl, and Fukushima.
Yes, I fully understand that over the long term coal plants cause many more deaths, and of course climate change is an order of magnitude worse, eventually. The issue is that human fears aren't based so much on "the long term" or "eventualities". When nuclear power plants failed, they had the unfortunate side effect of making hundreds or thousands of square miles uninhabitable for generations. The fact that societies demand heavy regulation for something that can fail so spectacularly isn't some underhanded conspiracy.
Just look at France, the country with probably the most successful wide-scale deployment of nuclear. They are rightfully proud of their nuclear industry there, but they are not a huge country (significantly smaller than Texas), and thus understand the importance of regulation to prevent any disasters. Electricity there is relatively cheap compared to the rest of Western Europe but still considerably higher than the US average.
And the fears from Chernobyl was MOSTLY irrational.
The reason for the extreme fears that are generated from even very moderate spills from nuclear plants comes in part from the association with nuclear bombs and in part from fear of the unknown.
A lot (if not most) people shut their rational thinking off when the word "nuclear" is used, even those who SHOULD understand that a lot more people die from coal and gas plants EVERY YEAR than have died from nuclear energy throughout history.
Indeed, the safety level at Chernobyl may have been atrocious. But so was the coal industry in the USSR. Indeed, even if just considering the USSR, the death toll from coal alone caused a similar number of deaths (or a bit more) than the deaths caused by Chernobyl EVERY YEAR [1].
[1] https://www.science.org/doi/pdf/10.1126/science.238.4823.11....
I'm so tired of hearing about how regulation is this magic salve that saves everything. Regulation is what caused Chernobyl. Soviet regulations mandated that the flawed RBMK reactor design be used. They knew it was flawed and they forced people to use it anyway. Because that's what government does. There's a similar story in western countries, where it hasn't been feasible to use better designs due to antiquated government regulations, it's just no one here has screwed up as badly as the Soviets did.
Here's an MIT study that dug into the reasons behind high nuclear construction cost. They found that regulatory burdens were only responsible for 30% of cost increases. Most of the cost overruns were because of needing to adapt the design of the nuclear plant to the site.
https://news.mit.edu/2020/reasons-nuclear-overruns-1118
Now, you can criticize the methodology of that study, but then you have to bring your own study that shows precisely which regulatory burdens are causing these cost overruns, and which of those are in excess. Is it in excess that we have strict workplace safety regulation now? Is it in excess that we demand reactor containment vessels to prevent meltdowns from contaminating ground water supplies? In order to make a good red tape argument I expect detail in what is excess regulation, and I've never seen that.
Besides, if "red tape" and fossil industry marketing was really the cause of unpopularity, and the business case for nuclear was real when installing plants at scale, you would see Russia and China have the majority of their electricity production from nuclear power.
- Russia is the most pro-nuclear country in the world, and even they didn't get past 20% of electricity share. They claim levelized cost for nuclear on the order of that of solar and wind, but I am very skeptical of that number, and anyone who knows anything about the Russian government's relation to the truth will understand why. When they build nuclear plants in other countries (e.g. the bangladesh project) they are not that cheap.
- China sits at low single digit percentages of nuclear share, with a levelized cost that is significantly higher than Russia's and well above that of solar and wind. While they're planning to grow the nuclear share they assume it will be based on new nuclear technology that changes the business case.
Both Russia and China can rely on cheap skilled labor to bring down costs, a luxury western countries do not have.
And this is ultimately the issue: the nuclear industry has been promising designs that bring down costs for over half a century, and they have never delivered on that promise. It's all smoke and mirrors distracting from the fact that building nuclear plants is inherently freaking expensive. Maybe AI can help us design better and cheaper nuclear power plants, but as of today there is no proven nuclear plant design that is economical to build, and that is ultimately why you see so little new nuclear plant construction in the west.
Not France?
> France derives about 70% of its electricity from nuclear energy, due to a long-standing policy based on energy security.
— https://world-nuclear.org/information-library/country-profil...
[0] https://www.mdpi.com/1996-1073/10/12/2169 - Fig 3
This is conjecture. If you wanted to establish this, you would have to show that cost of (skilled) labor was unchanging or negligible.
It is also important to consider that nuclear power deaths/damages are much more localized and traceable than excess deaths from air pollution, and thus much less acceptable to the voting population-- you could argue that this should not make any difference (I disagree), but I don't want to digress here too much.
> we're all lucky that the Chinese chose a different tack to the West's policy of energy failure
What do you believe that is? Because from my point of view, China generates a negligible amount of electricity from nuclear power (<5%), this is not going to change within the next decades, and the main "purpose" from what I can tell is to in-house reactor/turbine know-how (instead of relying on Alstom/Siemesn).
> It is also important to consider that nuclear power deaths/damages...
Maybe you can answer this for me - what deaths and damages? So far I've never been able to pin down any actual death or damage to a nuclear meltdown. I'm sure there are some, but most of the actual attempts to quantify it require appealing to hypothetical deaths and damages that no-one can specifically point to, or tiny numbers that are irrelevant to industrial policy.
I know people who lived in a town next to a lead-zinc mine. That appears to be about as bad as a nuclear crisis from what I can gather and it doesn't seem to be causing anyone undue stress. We're still using lead and zinc. People still live in the town.
> What do you believe that is?
They're building reactors. https://en.wikipedia.org/wiki/List_of_commercial_nuclear_rea... is a happy tale of new and planned plants.
Some of them are really cool too, there is one by the Gobi desert, apparently to prove that they don't need to use water as a coolant.
> Those are constant dollars. Are you claiming that the real cost of labour went up 4- to 8-fold in the nuclear industry?
I'm not saying that is the only effect. But if you want to blame "onerous regulations" for nuclear power being so expensive, then you have to at least put bounds on other factors driving cost (and you also have to show those relaxed regulations would not have led to a significant amount of additional incidents; the current rate of 2 meltdowns for ~500ish reactor sites is already problematic enough).
I personally think that collective standards regarding risk and pollution have risen significantly since the early days of nuclear power, and a lot of things that were done back then would be considered unacceptably reckless/negligient nowadays (=> Hanford became a superfund site for a reason), so claiming that those historical costs are repeatable seems very dubious in the first place to me.
Ok, so say they don't evacuate the city. What is the actual risk here that we're talking about?
> I'm not saying that is the only effect.
What other effect are you considering? The materials aren't getting that more expensive and the labour will cost similar too. There aren't a lot of options left apart from regulatory changes.
> I personally think that collective standards regarding risk and pollution have risen significantly...
Then do you potentially think that the reason the price is rising is because of regulation? Because this whole paragraph reads like a justification for the costs incurred due to regulation. I'm not seeing what your complaint is here pinning the inverted learning curve on the regulators - you seem to be saying that is what happened and it is reasonable in this paragraph.
The risk is that hundreds of people avoidably die of cancer over the following decades, and that you lose an immense amount of trust and credibility in government. This is not easy to de-risk because there are a lot of potential mechanisms for bio-accumulation/concentration, and by the time you find out what those exact mechanisms are empirically, the damage might already be done (and by then its too late, and people are no longer gonna trust anything you tell them).
I'm not arguing that the Fukushima evacuation was 100% the right call to make even in hindsight, but it was most certainly a defensible position (even in retrospect).
> Then do you potentially think that the reason the price is rising is because of regulation?
Strongly disagree with the framing. My opinion is that most early reactors did not meet modern safety/pollution expectations and quite frequently resulted in a toxic mess that still requires cleanup today (Sellafield would be a non-US example).
Despite all regulations, the safety track record (2 meltdowns at 500ish reactor sites) is arguably really poor, especially when reactor meltdowns had been portrayed as "virtually impossible" by early proponents.
> What other effect are you considering?
The very same effects that make bespoke heavy machinery comparatively expensive nowadays in general:
- Low benefit from industrial automation
- Mechanical engineering sectors have shrunk in the west (=> parts/subassemblies and qualified labor more expensive)
- Generally higher wages
Those apply much less to renewables because those are less labor intensive and benefit from economies of scale more (by design). Its also much easier to benefit from cheap foreign labor with those (especially solar).
As a sanity check:
I would always expect coal power plants to stay significantly cheaper than nuclear power (when disregarding CO2/fuel cost). But even in that setting, modern coal power (just like nuclear power) already struggles to compete with solar/wind on levelized cost basis today (and trends are very clear!)
So even if we dropped all nuclear regulation today completely (with disastrous likely effects) then the absolute best we could hope for would be to get competitive (on levelized cost) with the most expensive forms of renewables (offshore wind and rooftop solar). Given that it would likely take decades to act on such a plan, all the risks (and current downward trends on renewables and storage costs) chasing that approach seems utterly pointless to me.
Thus I believe the current nuclear approach (build a few reactors here and there so the know-how is not lost completely), which is exactly what both China and the US are doing, is perfectly sensible.
For long-term displacement, many people (mostly sick and elderly) died at an increased rate while in temporary housing and shelters. Degraded living conditions and separation from support networks are likely contributing factors. As of 27 February 2017, the Fukushima prefecture government counted 2,129 "disaster-related deaths" in the prefecture. This value exceeds the number that have died in Fukushima prefecture directly from the earthquake and tsunami. .... As of the year 2016, among those deaths, 1,368 have been listed as "related to the nuclear power plant" according to media analysis
~ https://en.wikipedia.org/wiki/Fukushima_nuclear_accident_cas...
Do you actually mean 100s over decades? Given that the evacuations from Fukushima caused thousands (well, thousand - but we're talking the highest hundreds before we get to thousands) of deaths in years, it seems like the risk of nuclear meltdowns would be completely acceptable. The government forced people to accept higher risk because bureaucrats panicked and it seems that was acceptable. What is the problem with a meltdown supposed to be if more damage is done just for theatre?
You're proposing people respond to a threat that isn't really there. Hundreds of people routinely die over decades from all sorts of industrial processes. A once in a generation risk that somewhere on the globe will suffer, maybe, 100s of deaths over the course of a few decades is nothing. Coal is still worse, is everywhere and people accept that. Is this the limit of the damage you're worried about?
Many things are cheap when you ignore externalities.
It’ll never be too cheap to meter, but electricity will get much cheaper over the coming decades, and so will synthetic hydrocarbons on the back of it.
There are better options, and at scale they're literally capable of producing electricity that literally is too cheap to meter.
The reasons they haven't been built at scale are purely political.
Today's AI is computing's equivalent of nuclear energy - clumsy, centralised, crude, industrial, extractive, and massively overhyped and overpriced.
Real AI would be the step after that - distributed, decentralised, reliable, collaborative, free in all senses of the word.
(that excludes a brief period when I camped with a solar panel)
but here is some data bro! https://fred.stlouisfed.org/series/APU000072610
also this is weird i thought electricity prices only get cheaper? https://fred.stlouisfed.org/series/APU000072610
Microsoft: https://news.ycombinator.com/item?id=41601443
Google: https://news.ycombinator.com/item?id=41840769
> The deal would help enable a revival of Unit 1 of the five-decades-old facility in Pennsylvania that was retired in 2019 due to economic reasons.
i wonder why they stopped producing energy in 2019 even though energy prices have gone up over the five decades?
Another way for intelligence to get too cheap to meter is for the cost to fall so low it becomes hyperabundant. If you were to, for instance, take AI2027 as a benchmark and think ultimately we'll achieve something like the equivalent of John von Neumann in a box with a 2T dense equivalent parameter model and it will match such a Nobel prize winner's productivity when running inference at say 15 tokens a second (as fast as people can read) then you only need in principle 60 teraflops of AI infernce compute, which is roughly 2x the current Apple Neural Engine. So plausibly by the time you get to the 2030s, every laptop, smartphone, etc will be easily able to run models as powerful as the smartest people.
Somewhat longer term, I'm sure Altman expects the entire process to be automated and for the computational efficiency to rise significantly. If you take recent estimates from various players in the reversible computing space, you'd guesstimate that you ought to be able to do 60tflops by the late 2030s using under 0.1W or ~1kWh/yr which Helion could produce for ~1¢. I do feel like 1 year of cognitive labor from the smartest person a penny or two renders intelligence too cheap to meter out on a per-hour basis.
Not sure if wishful thinking trying to LARP-manifest this future into being, or just more unfalsifiable thinking where we can always be said to be past the event horizon and near to the singularity, given sufficiently underwhelming definitions of "event horizon" and "nearness."
"Those who can make you believe absurdities, can make you commit atrocities." - Voltaire
Of course, progress could stall out, but we appear to have sufficient compute to do anything a human brain can do, and in areas, AIs are already far better than humans. With the amount of brain power working in, and capital pouring into, this area, including work on improving algorithms, I think this essay is fundamentally correct that the takeoff has started.
You could say the same thing about a CPU from 40 years ago - they can do math far better than humans. The problem is that there are some very simple problems that LLMs can’t seem to reliably solve that a child easily could, and this shows that there likely isn’t actual intelligence going on under the hood.
I think LLMs are more like human simulators rather than actual intelligent agents. In other words, they can’t know or extrapolate more knowledge than the source material gives them, meaning they could never become more intelligent than humans. They’re like a very efficient search engine of existing human knowledge.
Can anyone give me an example of any true breakthrough that was generated by an LLM?
40 years ago we were clearly compute bound. Today, I think it's fairly clear we are not; if there is anything a human can do that an AI can't, it's because we lack the algorithms, not the compute.
So the question becomes, now that we have sufficient compute capacity, how long do you think it will take the army of intelligent creative humans (comp sci PhDs, and now accelerate by AI assistance) to develop the algorithmic improvements to take AI from LLMs to something human level?
Nobody knows the answer to the above, and I could be very wrong, but my money would bet on it being <30 years, if not dramatically sooner (my money is on under 10).
It seems to me like the building blocks are all here. Computers can now see, process scenes in real time, move through the world as robots, speak and converse with humans in real time, use tools, create images (imagine?), and so forth. Work is continuing to give LLMs memory, expanded context, and other improvements. As those areas all get improved on, tied together, recursively improved, etc., at some point I think it will be hard to argue it is not intelligence.
Where we are with LLMs is Kitty Hawk. The world now knows that flight (true human level intelligence) is possible and within reach, and I strongly believe the progress from here on out will continue to be rapid and extreme.
This assumes that the eventual breakthroughs start from something like LLMs. It's just as likely or more that LLMs are an evolutionary dead end or wrong turn, and whatever leads to AGI is completely unrelated. I agree that we are no longer compute bound, but that doesn't say anything about any of the other requirements.