This is my concern with AI in general. Cost, both real and monetary. Right now Microsoft and VCs are dumping money into AI operation to help with growth and adoption. What happens when AI's business focus moves from cost to grow to cost to serve? Will all these business who integrated in AI suddenly be saddled with huge bills? What if your product depends on AI, and suddenly is not profitable to operate? Anecdotally I have already seen people pull back AI features, because it turned out to be too expensive to serve in the long run.
I already pay for a GPT subscription, and its reliability is one of the worst of any product I pay for. The novelty keeps me paying, but I can't imagine building a business on it.
> if the people providing the AI demand the same money as the workers it replaces, it doesn't seem like society actually benefits
Those people no longer have jobs. That sounds bad, but consider they can now do something else. (Ad infinitum this is obviously a problem. But the history of technological development provides cause for optimism in the long run.)
The majority of Americans live paycheck to paycheck. On average, American's may have sixth months of savings, but that is due to the long tail of wealth distribution. Only about 1 out of 4 of them actually have six months or more of savings.
> the history of technological development provides cause for optimism in the long run
I think it actually provides cause for pessimism: in the past those people had other kinds of jobs to move to. But this AI revolution makes it much harder to move to another job other than a menial one because a lot of the lower level office jobs are affected all at once. This creates a lot of downward pressure on fields that were already paying peanuts and where employers have realized they can now squeeze even further, either by cutting wages directly or by having more desperate entrants in the race to the bottom.
Going from agricultural work into technology was an improvement, going from office work to unskilled labor is a regression. Upward mobility is limited because there is less room there anyway and there too there will be more competition for fewer jobs.
So for the moment I don't really see the upside on a societal scale, even if for some individuals there are upsides.
> this AI revolution makes it much harder to move to another job other than a menial one because a lot of the lower level office jobs are affected all at once
If we automate away administration, there is a bonanza to be had. Every person would in essence be a start-up team. That's enough surplus to figure out a transition. I'm not optimistic about every political system finding the solution. But some will, and then it slowly spreads.
Most people don't want to be start-ups. Media is overbiased towards leaders and makers. But the average person just wants to show up, get paid, and go home. They don't want to have to figure out how to constantly reinvent themselves to be marketable.
Most people don’t want to do work. The point is there will be terrific surpluses generated at every level of society. That gives those people choices. In some societies, they’ll horde it. But in resilient ones, they’ll recognise the long-term collective interest in ensuring everyone who can make does.
The number of people that want to be part of a start-up is limited, the number of people that want the responsibility that comes with being an entrepreneur is further limited. Those people that would like to be entrepreneurs are entrepreneurs, the rest is more than happy to just have a job and a stable life. Risk appetite and willingness to hyperfocus on one thing at the expense of the rest, including quality of life is something that varies widely from one individual to another.
HN is not an 'average' in this sense at all, more likely an extreme outlier. This is also why the 'gig economy' is such a huge step back.
> Those people that would like to be entrepreneurs are entrepreneurs, the rest is more than happy to just have a job and a stable life
The 9 to 5 job was invented alongside the Industrial Revolution. (And clocks.) Before that, many civilisations were collections of entrepreneurial households. (Plus slaves/serfs/servants.)
The point is civilisation adapts. But the long run, in making some people more productive, has historically been everyone getting richer.
It’s much easier to be entrepreneurs when you aren’t competing with literally millions of people world wide. The bar is so much higher now. If you wanted to be a blacksmith you were only competing with people within a few day walking distance. If I want to make bespoke software I’m competing with multinational conglomerates.
> Before that, many civilisations were collections of entrepreneurial households. (Plus slaves/serfs/servants.)
Those were the exceptions, not the rules, the slaves, serfs and servants were the bulk and what is happening now is that we are re-creating the conditions where lots of people will have nothing to offer but their physical labor, in that sense it is the reverse of the industrial revolution. But couple AI with robotics and you might not need those people at all. What do you propose to do with them? What about the millions of translators, truck drivers, copywriters cab drivers, couriers and so on?
If you propose they become entrepreneurs in what domain should this happen? And what will safeguard those domains from being usurped in turn?
It's interesting how the fact that civilization has adapted to date gets taken as proof that it will always work but that's faulty logic: this time it may not work and even if it worked for society it most definitely didn't work for all of the individuals in it. And this time around it may not work for the majority of the individuals in it.
> the slaves, serfs and servants were the bulk and what is happening now is that we are re-creating the conditions where lots of people will have nothing to offer but their physical labor
This was the exact argument made during the Industrial Revolution. Keep in mind that a minority of workers today are in white-collar jobs. We're over a century out from mechanising physical labor, and it's still strongly present.
> what about the millions of translators, truck drivers, copywriters cab drivers, couriers and so on?
Drafting spreadsheets by hand was a profession up to teh 1980s. Same for reams of printing and document-couriering services. People adapted.
> If you propose they become entrepreneurs in what domain should this happen?
Idk, launch a florist or ski instructing or tour guiding service. Travelling chef. There are so many talented people with zero knack for administration stuck in service jobs.
> this time it may not work and even if it worked for society it most definitely didn't work for all of the individuals in it. And this time around it may not work for the majority of the individuals in it.
Not using precedent as proof. Just saying there is precedence for technological revolutions and this very concern. The fact that it's gone pretty much one way elevates the burden of proof for those preaching doom and gloom.
Another observation: the socieities that best distributed the gains in a way that was win-win were those who approached it with optimism.
> it may not work for the majority of the individuals in it
Sure. I'm not saying the transition won't be hard. But it's not avoidable. And in the long run, precedence shows it should (or at the very least, can) work out. Having excess production and a labour surplus is a champagne problem. That doesn't mean one can't fuck it up.
> This was the exact argument made during the Industrial Revolution. Keep in mind that a minority of workers today are in white-collar jobs. We're over a century out from mechanising physical labor, and it's still strongly present.
Yes, but then it was a shift from one kind of labor to another. Now it is a shift from some kind of labor into nothing.
> Drafting spreadsheets by hand was a profession up to teh 1980s. Same for reams of printing and document-couriering services. People adapted.
That's a nice mantra but it doesn't put food on any tables. People adapt to large scale war as well, mostly by dying and people adapt to famine, earthquakes and floods as well, mostly by dying. Whose to say that massive unemployment because there literally is no longer enough work to go around (which technically is already the case!) the income streams of which power all of our collective economies is something that we can 'survive' in any form? You are so sure because it worked in the past but that doesn't offer any guarantees for the future at all. That's the same kind of reasoning that would have someone endlessly pull the trigger during Russian Roulette: it worked so far! Until it doesn't...
> Idk, launch a florist or ski instructing or tour guiding service. Travelling chef. There are so many talented people with zero knack for administration stuck in service jobs.
There is only so much demand for florists, ski instructors, traveling chefs or tour guides and those jobs are mostly taken.
The reason those people are in administrative jobs is because that's where the money is. If that source of income disappears they don't just evaporate, they are now 'unstuck' from their source of income, that doesn't change their needs one bit (and in many ways increases those needs, including psychological needs).
> Not using precedent as proof. Just saying there is precedence for technological revolutions and this very concern. The fact that it's gone pretty much one way elevates the burden of proof for those preaching doom and gloom.
See Russian Roulette analogy above. It's survivorship bias warmed over.
> Another observation: the socieities that best distributed the gains in a way that was win-win were those who approached it with optimism.
Yes, optimism all the way to the polluted and destabilized world that we live in today. We will be dealing with the consequences of that revolution for the next 20 decades or so and that's assuming that the next one isn't going to do us in. The industrial revolution worked out for some parts of the world. But others were left behind without a moment's thought (well, ok, an occasional tear was shed). The number of people that this revolution leaves behind could very well be orders of magnitude larger.
>> it may not work for the majority of the individuals in it
> Sure. I'm not saying the transition won't be hard. But it's not avoidable. And in the long run, precedence shows it should (or at the very least, can) work out. Having excess production and a labour surplus is a champagne problem. That doesn't mean one can't fuck it up.
I'm not sure 'won't be hard' is strong enough and I'm not sure it isn't avoidable, regardless I'm pretty sure that as long as we aren't able to clean up our present day messes that we probably shouldn't be opening more Pandora's boxes unless we have a plan on how we're going to deal with the possible aftermath. Anything less would be - especially given our track record of these things to date - utterly irresponsible.
I know quite a few school age kids. If there is one scary and common thread that runs through what I keep hearing it is that they have universal apathy regarding their future, between 'AI', climate change and various wars why study, why prepare for a world that's changing too fast to keep up with? You may as well enjoy...
What's the basis for casting technological revolutions as a game of Russian roulette? Of course your model will predict doom if you hard code that in.
> industrial revolution worked out for some parts of the world
I mean, sure. If you think the world would be better off had we never industrialized or invented agriculture, I guess there's a lot to be depressed about.
> not sure it isn't avoidable
The tech has military applications. It's not avoidable.
> as long as we aren't able to clean up our present day messes that we probably shouldn't be opening more Pandora's boxes unless we have a plan on how we're going to deal with the possible aftermath
You don't know what's in the box until you open it. We wouldn't have wind farms and solar panels (as efficient as they are) without computers. If we'd listened to the people who said we should stop digitising to save jobs we would be worse off vis-à-vis climate change.
> they have universal apathy regarding their future, between 'AI', climate change and various wars why study, why prepare for a world that's changing too fast to keep up with?
Where are these kids geographically? The brighter ones I know are brimming with optimism.
> unbounded optimism is likely to lead to some massive level of disappointment
By definition, right?
I'm not saying we won't be disappointed. But optimism with respect to technological revolutions tends to lead to a better place than fear and pessimism. Even if that better place has its disappointments.
I envy you your optimism. Meanwhile, I have one person here who is out of a job because 'automatic translation is good enough' (15 years of study down the drain) and a bunch of kids who have absolutely no idea on why they would put in effort in school given the imminent advent of AI and the short best-before dates on the kind of jobs that interest them.
If you have any good advice that transcends 'become an entrepreneur' and 'deal with it' then I'm all ears. Because this is no longer a theoretical issue for me.
> Where are these kids geographically?
NL.
> The brighter ones I know are brimming with optimism.
They're anything but stupid, in fact that's part of the problem, they're clever enough to see which way the wind blows. If they were less smart it would be less of a problem.
I think the critical skill for everyone to have now is "spot the bullshit" which can take different forms in different media and domains. Whenever the topic of AI comes up in family discussions I let my teens know just how much BS companies are willing to generate due to the investment dollars pouring in. Hence, AI is over-hyped for the most part, although unfortunately for your friend there are several automatic translation tools that just work.
However, if I ran a business where we needed to translate documents I still wouldn't trust that to an automatic tool. AI is seriously flawed, hopefully eventually more people will realize that before it buries the internet in layers of garbage.
To summarize, my children are mostly indifferent to the advent of AI because I always tell them it's complete trash.
But that's not true though, it isn't complete trash. Some of it is trash, some of it is nothing short of amazing. And the bit that is amazing, when applied at scale and properly weaponized already has the potential to transform our world in many ways.
> The 9 to 5 job was invented alongside the Industrial Revolution.
Industrial revolution enable some entrepreneurial households - that were few of them anyway - to move up. Most people were on that 'serfs' scale.
8 hour workday is regulation 'invented' as reaction to Laissez-faire of the Industrial Revolution.
It can help if there are simply not enough workers available and it's used to boost productivity of the existing workers.
through in most cases it probably will still not work/help society
I.e. from what I heard public defendants in the US are notoriously overworked, if an AI could help them auto prepare all the simple straight forward cases they could do a better job. But then this is also a good example for why it probably still will not work out well. As no one will pay for that tool or they will pay for it but then expect so many more cases to be handled that it might get worse instead of better.
first evidence that OpenAI is seriously lowballing it's tech bills and unless there's serious GPU efficiency gains, they're doing ye olde uber dances trying to get their consumers locked in before the price hikes start.
Ah we've now reached the "We're ready to go except for the magical development in another domain that's already resisted decades of concerted research and is absolutely necessary for what we do to actually make sense" stage. Good to know.
Record time compared to the crypto space, "all we need is infinite data storage with instant recall. Then we can store everything on the chain, and verify its authenticity!"
The AI winter is coming, and it's going to be bad. So many promises have been made, to the extent that the average person thinks that AGI is just around the corner.
Only at night. Build 3 data centra and you can provide full solar power to at least one of them at any time, just shift the workload to the resource with the power
Or base your DC in a location with lots of other renewable energy
Yes, and there's also a stealth way to capture it if your planet has any type of gaseous atmosphere: take advantage of air currents that arise from the repeated heating and cooling of your atmosphere using a wind turbine.
> Energy from nuclear fusion is available today – you just need solar panels to capture it
Nitpick: stellar fusion is fundamentally different from thermonuclear fusion.
Proton-proton fusion is mediated by the weak force [1][2]. A lump of the Sun radiates less power than your body. It works because there is a lot of Sun. Thermonuclear fusion is far more energetic because it's mediated by the strong force.
Yes, but the whole reason that works is because a chain of such devices is still stable. A chain of analog components will just output random noise if it gets long enough. That's the main reason why digital won out over analog: repeatability even with long chains of computation.
We have companies like Google and Microsoft who have the ability to build or create power generation for things like this. I just think it's interesting that Microsoft could be energy self sufficient by building generation next to a datacenter.
at least out west they build near power generation already (usually next to hydro dams). At least that is what I remember them doing, I could be a few years out of date.
Amazon is a huge investor in green energy, both solar and wind. Google also buys solar. Colocated energy is great in theory, but doesn't always scale. That is why most companies take an offset approach.
One should be suspicious of ulterior motives when the CEO of an AI company makes a claim like this.
On one hand, LLMs do require significant amounts of compute to train. But the other hand, if you amortize training costs across all user sessions, is it really that big a deal? And that’s not even factoring in Moore’s law and incremental improvements to model training efficiency.
Data centers cost 2% of electricity (Statistics from a few years ago).
AI inference is so costly at scale, one can easily see data centers start using 4% of total electricity, and in the next decade 8%. That will start to have severe effects on the power grid, basically require planning many years in advance to setup new power plants and such.
"Moore's law and incremental improvements' are irrelevant in the face of scaling laws. Since we aren't at AGI yet, every extra bit of compute will be dumped back into scaling the models and improving performance.
yes, at least as long as you constantly develop new AI models
and you still need to run the models, and e.g. for GPT4 that is alone already non trivial (energy cost/compute wise)
through for small LLMs if they are not run too much it might be not that bad
---
Generally I would always look for ulterior motives for any "relevant" public statement Sam Altman makes. As history has shown there often seems to be some (through in that usage "ulterior" has a bit too much of a "bad"/"evil" undertone).
To cut it short he seem to be invested in some Nuclear Fusion company, which is one of the potential ways to "solve" that problem. Another potential way is to use smaller LLMs but smaller LLMs can also be potentially a way how OpenAI loses their dominant position, as there is a much smaller barrier for training them.
He's probably trying to build out a vertically integrated AI conglomerate spanning power, chip fabs, software, etc. And he's great at marketing in the meantime
I think both the OpenAI and the Sam Altman brand sustained some damage. He may be great at marketing but now that the gun is on the table and the cloth is withdrawn we'll see how much success he has in the future with trying to wrap people around his fingers.
The size some of the large language models have is just insane. To put it into context GPT3 is a Large Language Model (LLM) but GPT4 has like (handwaveing) 5+ times as many parameters. This means the energy cost is also at least that much larger for inference.
And if we look at training instead of inference it's a quite a bit more complicated (each iteration is more costly, more parameter can also require more iterations, but at the same time you might need to spend less time in an area of increasingly diminishing returns per iteration), but if we go with GPT3 vs. GPT4 the 5x+ increase of parameters lead to a 10x+ increase in training cost (of which a huge part is energy cost, through also amortized hardware cost; ~10M$ to >100M$).
Additionally there are various analysis steps you might do when creating new models which can be hugely (energy) costly.
And that is with GPT4 and with OpenAI any major increase in version seem to come with a major increase in parameter size so with that trend we are looking at energy bills in the (potential many) hundreds of million US dollar for training alone.
Another thing wrt. inference cost is that with my limited understanding currently the best way to reach AGI and also a lot of other tasks is to run multiple models in tandem. Through this model might be domain adoptions of the same model, so not twice the training cost.
---
Domain Adaption == take a trained model and then train it a bit more to "specialize" it on a specific task, potentially adding additional knowledge etc. While you can try to do so just with prompt engineering there a larger prompt comes at a larger cost and a higher chance for unexpected failure, so by in a certain way "burning in" some additional behavior, knowledge etc. you can get nice benefit.
Well, if the AI is good enough then ask it to design one.
Joking aside: the options are to turn it off or to charge what it should cost and given that no matter what it costs and no matter what the source if it uses too much energy the better solution would be to make it use less energy rather than to look for a breakthrough energy source because that also reduces the climate impact.
Well actually: "In a twist, Microsoft is experimenting with generative artificial intelligence to see if AI could help streamline the [nuclear power] approval process, according to Microsoft executives." [0]
can't read the article but if it's roughly about what I would expect (and have seen in other fields) the idea is roughly:
The final proof reading and decision still is done by humans, but before that automatic feedback AI is used to find things which need to be changed during planing, certification and highlight potential problem. I.e. eliminate as many rounds of back and force between agency and plant builder and in turn save a lot of time.
Through if the final proof reading is also done by AI or relies to much on AI this is a huge issue. LLMs fundamentally to their design sometimes (in the future maybe rarely) make mistakes in forms which for humans seem arbitrary, random and absurd (in turn they tend to also not do some of the mistakes humans tend to do). The problem is even if this mistakes become rare they always can be fundamental safety issues.
It's unfortunate that no math is ever done in these stories.
If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W, which gives you almost 250 MW for just the GPUs. That sounds like a lot, until you realize that a single large power plant is measured in Gigawatts. All of Google's data centers combined are O(10 GW) which are matched with renewable power offsets [1].
Importantly, the world installed >500 Gigawatts of renewable energy in 2023 [2], mostly driven by PV Solar in China. The amount of potential solar and wind and other renewable-ish (hydro) outstrips even a 10x'ing of a lot of these numbers. But even for a single site, dams like Three Gorges are >20 GW.
There are real efficiency and scale challenges in doing AI in a single, large site. But existing power generation systems deliver plenty of power.
The big point of course is that there is massive asymmetry between training and inference and that even inference at scale is going to require massive amounts of energy and that likely OpenAI's business model isn't viable at scale. It works right now because they have capital to burn but when the music stops it may well turn out that their model isn't sustainable at all.
Efficiency gains should come first, long before they start looking at alternative energy sources.
Energy production capacity in the US is relatively flat for the last couple of decades. The renewable installation is offsetting the decommission of coal. The capacity installed in China is not really accessible to Open AI due to recent security competition (both export restrictions on AI and a desire to import less energy). The capital costs of power are also quite high so I think he is pretty accurate considering the expectations of a startup to hockey stick.
> This report also analyzes prospective generation capacity in four categories — under construction, permitted, application pending, and proposed. More than 466,000 MW of new generation capacity is under development in the United States — a 13% increase over 2022. Sixty-one percent of capacity most likely to come online, permitted plants and plants that are under construction, are in solar.
China's growth in power capacity is non-trivially due to increasing demand. If the US or Europe or wherever suddenly wanted to build XXX GW per year, they could (modulo bureaucracy, which is very real).
The US or EU could likely do it, yes. If someone is willing to put up the capital to do it. The point of Sam Altman's statement is that he wants someone to do it.
Just to clarify, PUE of 1.1 means 10% power waste not spent on computing. However, Google datacenters are extremely efficient, and they design everything end-to-end for efficiency. A normal datacenter with less efficient cooling, less efficient power delivery, less efficient buildings, etc. easily adds up to a lot more wasted power.
Facebook is not alone, and there is growth. Also cooling is to be taken account of.
And third: renewables need to be associated with its backup like hydro/step or ... batteries which cost a lot. Gas can't be taken in as it's not CO2-free. All that unless training and inference happen when there's the corresponding wind and sun shining. And I'm not seeing that happening right now.
This is why we need to focus on smaller, more efficient models.
A lot of the 13 and 30bn parameter models are actually quite good, maybe not GPT4 but it can run on my M1 MBP without an issue, and a bit faster than GPT4 anyway.
I don't need a chat interface that is amazing at all things at once, I need it to be great at 1 subject at a time.
> we need better ways to store energy from solar power.
Why? I understand that for a normal website you always want to use the closest server possible, but for queries that take 30 seconds on average anyway, why does it matter if you're adding an extra 250ms to get it from the opposite side of the world?
If AI can legitimately replace human labor, why would anyone care if it takes an extra few milliseconds?
Is this the new PR playbook? Claim to be building something big and dangerous and ready to ruin everything if everyone doesn't drop everything and pay attention to me?
Bitcoin is still wasting gigawatts every year, but later blockchains are literally a million times more efficient. If the current generations of AI turn out to be useful, maybe someone will figure it out an efficiency optimization.
People and governments really need to start asking if the results of the current generation of FAANG-size LLMs are worth the costs, in additional chips, power, and cooling water -- especially in locations that are constrained for any of those resources. Are we getting ten times better Google results for ten times more energy? I don't think so.
What kind of 'AI' is he talking about? The AGI kind? The Superintelligence kind? The chatbot which can't easily be ridiculed using esoteric word incantations kind?
I want one which can instantly calculate numberwang[0] and get it right every time
That's the Big Tech playbook: go towards technology so expensive to run that competition is impossible. Tech that cannot be replicated. And of course the main fuel is our work, our content.
93 comments
[ 2.3 ms ] story [ 156 ms ] threadI already pay for a GPT subscription, and its reliability is one of the worst of any product I pay for. The novelty keeps me paying, but I can't imagine building a business on it.
Employers will save health, logistics, HR, etc.
Governments will have to pay for unemployment
Just the same as always - privatize the gains
Those people no longer have jobs. That sounds bad, but consider they can now do something else. (Ad infinitum this is obviously a problem. But the history of technological development provides cause for optimism in the long run.)
That's 6 months more than during the agricultural and industrial revolutions. The world isn't great, but it's getting better in the long run.
I think it actually provides cause for pessimism: in the past those people had other kinds of jobs to move to. But this AI revolution makes it much harder to move to another job other than a menial one because a lot of the lower level office jobs are affected all at once. This creates a lot of downward pressure on fields that were already paying peanuts and where employers have realized they can now squeeze even further, either by cutting wages directly or by having more desperate entrants in the race to the bottom.
Going from agricultural work into technology was an improvement, going from office work to unskilled labor is a regression. Upward mobility is limited because there is less room there anyway and there too there will be more competition for fewer jobs.
So for the moment I don't really see the upside on a societal scale, even if for some individuals there are upsides.
If we automate away administration, there is a bonanza to be had. Every person would in essence be a start-up team. That's enough surplus to figure out a transition. I'm not optimistic about every political system finding the solution. But some will, and then it slowly spreads.
Most people don’t want to do work. The point is there will be terrific surpluses generated at every level of society. That gives those people choices. In some societies, they’ll horde it. But in resilient ones, they’ll recognise the long-term collective interest in ensuring everyone who can make does.
It is important to remember that innovation doesn't raise all boats, even if it raises the average boat. Some will sink, along with their crew.
We comfortably laugh at the resistant luddites, but many of them died impoverished, were shot, or hung.
That shouldn't be a condemnation of innovation, but simply to point out that there are real winners and losers.
HN is not an 'average' in this sense at all, more likely an extreme outlier. This is also why the 'gig economy' is such a huge step back.
The 9 to 5 job was invented alongside the Industrial Revolution. (And clocks.) Before that, many civilisations were collections of entrepreneurial households. (Plus slaves/serfs/servants.)
The point is civilisation adapts. But the long run, in making some people more productive, has historically been everyone getting richer.
Those were the exceptions, not the rules, the slaves, serfs and servants were the bulk and what is happening now is that we are re-creating the conditions where lots of people will have nothing to offer but their physical labor, in that sense it is the reverse of the industrial revolution. But couple AI with robotics and you might not need those people at all. What do you propose to do with them? What about the millions of translators, truck drivers, copywriters cab drivers, couriers and so on?
If you propose they become entrepreneurs in what domain should this happen? And what will safeguard those domains from being usurped in turn?
It's interesting how the fact that civilization has adapted to date gets taken as proof that it will always work but that's faulty logic: this time it may not work and even if it worked for society it most definitely didn't work for all of the individuals in it. And this time around it may not work for the majority of the individuals in it.
This was the exact argument made during the Industrial Revolution. Keep in mind that a minority of workers today are in white-collar jobs. We're over a century out from mechanising physical labor, and it's still strongly present.
> what about the millions of translators, truck drivers, copywriters cab drivers, couriers and so on?
Drafting spreadsheets by hand was a profession up to teh 1980s. Same for reams of printing and document-couriering services. People adapted.
> If you propose they become entrepreneurs in what domain should this happen?
Idk, launch a florist or ski instructing or tour guiding service. Travelling chef. There are so many talented people with zero knack for administration stuck in service jobs.
> this time it may not work and even if it worked for society it most definitely didn't work for all of the individuals in it. And this time around it may not work for the majority of the individuals in it.
Not using precedent as proof. Just saying there is precedence for technological revolutions and this very concern. The fact that it's gone pretty much one way elevates the burden of proof for those preaching doom and gloom.
Another observation: the socieities that best distributed the gains in a way that was win-win were those who approached it with optimism.
> it may not work for the majority of the individuals in it
Sure. I'm not saying the transition won't be hard. But it's not avoidable. And in the long run, precedence shows it should (or at the very least, can) work out. Having excess production and a labour surplus is a champagne problem. That doesn't mean one can't fuck it up.
Yes, but then it was a shift from one kind of labor to another. Now it is a shift from some kind of labor into nothing.
> Drafting spreadsheets by hand was a profession up to teh 1980s. Same for reams of printing and document-couriering services. People adapted.
That's a nice mantra but it doesn't put food on any tables. People adapt to large scale war as well, mostly by dying and people adapt to famine, earthquakes and floods as well, mostly by dying. Whose to say that massive unemployment because there literally is no longer enough work to go around (which technically is already the case!) the income streams of which power all of our collective economies is something that we can 'survive' in any form? You are so sure because it worked in the past but that doesn't offer any guarantees for the future at all. That's the same kind of reasoning that would have someone endlessly pull the trigger during Russian Roulette: it worked so far! Until it doesn't...
> Idk, launch a florist or ski instructing or tour guiding service. Travelling chef. There are so many talented people with zero knack for administration stuck in service jobs.
There is only so much demand for florists, ski instructors, traveling chefs or tour guides and those jobs are mostly taken.
The reason those people are in administrative jobs is because that's where the money is. If that source of income disappears they don't just evaporate, they are now 'unstuck' from their source of income, that doesn't change their needs one bit (and in many ways increases those needs, including psychological needs).
> Not using precedent as proof. Just saying there is precedence for technological revolutions and this very concern. The fact that it's gone pretty much one way elevates the burden of proof for those preaching doom and gloom.
See Russian Roulette analogy above. It's survivorship bias warmed over.
> Another observation: the socieities that best distributed the gains in a way that was win-win were those who approached it with optimism.
Yes, optimism all the way to the polluted and destabilized world that we live in today. We will be dealing with the consequences of that revolution for the next 20 decades or so and that's assuming that the next one isn't going to do us in. The industrial revolution worked out for some parts of the world. But others were left behind without a moment's thought (well, ok, an occasional tear was shed). The number of people that this revolution leaves behind could very well be orders of magnitude larger.
>> it may not work for the majority of the individuals in it
> Sure. I'm not saying the transition won't be hard. But it's not avoidable. And in the long run, precedence shows it should (or at the very least, can) work out. Having excess production and a labour surplus is a champagne problem. That doesn't mean one can't fuck it up.
I'm not sure 'won't be hard' is strong enough and I'm not sure it isn't avoidable, regardless I'm pretty sure that as long as we aren't able to clean up our present day messes that we probably shouldn't be opening more Pandora's boxes unless we have a plan on how we're going to deal with the possible aftermath. Anything less would be - especially given our track record of these things to date - utterly irresponsible.
I know quite a few school age kids. If there is one scary and common thread that runs through what I keep hearing it is that they have universal apathy regarding their future, between 'AI', climate change and various wars why study, why prepare for a world that's changing too fast to keep up with? You may as well enjoy...
I'm arguing that this is far from a certainty. If anything, it seems to be a creature of the AI companies' leadership's marketing.
> reason those people are in administrative jobs is because that's where the money is
This doesn't preclude there being other things there's money in. Just consider how much cheaper education becomes without administrative bloat.
> See Russian Roulette analogy above. It's survivorship bias warmed over.
What's the basis for casting technological revolutions as a game of Russian roulette? Of course your model will predict doom if you hard code that in.
> industrial revolution worked out for some parts of the world
I mean, sure. If you think the world would be better off had we never industrialized or invented agriculture, I guess there's a lot to be depressed about.
> not sure it isn't avoidable
The tech has military applications. It's not avoidable.
> as long as we aren't able to clean up our present day messes that we probably shouldn't be opening more Pandora's boxes unless we have a plan on how we're going to deal with the possible aftermath
You don't know what's in the box until you open it. We wouldn't have wind farms and solar panels (as efficient as they are) without computers. If we'd listened to the people who said we should stop digitising to save jobs we would be worse off vis-à-vis climate change.
> they have universal apathy regarding their future, between 'AI', climate change and various wars why study, why prepare for a world that's changing too fast to keep up with?
Where are these kids geographically? The brighter ones I know are brimming with optimism.
> unbounded optimism is likely to lead to some massive level of disappointment
By definition, right?
I'm not saying we won't be disappointed. But optimism with respect to technological revolutions tends to lead to a better place than fear and pessimism. Even if that better place has its disappointments.
If you have any good advice that transcends 'become an entrepreneur' and 'deal with it' then I'm all ears. Because this is no longer a theoretical issue for me.
> Where are these kids geographically?
NL.
> The brighter ones I know are brimming with optimism.
They're anything but stupid, in fact that's part of the problem, they're clever enough to see which way the wind blows. If they were less smart it would be less of a problem.
However, if I ran a business where we needed to translate documents I still wouldn't trust that to an automatic tool. AI is seriously flawed, hopefully eventually more people will realize that before it buries the internet in layers of garbage.
To summarize, my children are mostly indifferent to the advent of AI because I always tell them it's complete trash.
Industrial revolution enable some entrepreneurial households - that were few of them anyway - to move up. Most people were on that 'serfs' scale. 8 hour workday is regulation 'invented' as reaction to Laissez-faire of the Industrial Revolution.
through in most cases it probably will still not work/help society
I.e. from what I heard public defendants in the US are notoriously overworked, if an AI could help them auto prepare all the simple straight forward cases they could do a better job. But then this is also a good example for why it probably still will not work out well. As no one will pay for that tool or they will pay for it but then expect so many more cases to be handled that it might get worse instead of better.
Or base your DC in a location with lots of other renewable energy
Also makes it easier to fight when it turns evil
Nitpick: stellar fusion is fundamentally different from thermonuclear fusion.
Proton-proton fusion is mediated by the weak force [1][2]. A lump of the Sun radiates less power than your body. It works because there is a lot of Sun. Thermonuclear fusion is far more energetic because it's mediated by the strong force.
[1] https://en.wikipedia.org/wiki/Proton–proton_chain
[2] https://en.wikipedia.org/wiki/Beta_plus_decay
- technically complex
- use cases uncertain
- except for crime/spam/other unpleasantness
- Nvidia is smiling
- cooking the planet
seems like it to me!
I have some legit usecases for Generative AI, it advanced my productivity across multiple learning areas by at least 2x.
Blockchain on the other hand, a pile of wannabe numbers in the air.
https://www.aboutamazon.com/news/sustainability/amazon-renew...
On one hand, LLMs do require significant amounts of compute to train. But the other hand, if you amortize training costs across all user sessions, is it really that big a deal? And that’s not even factoring in Moore’s law and incremental improvements to model training efficiency.
AI inference is so costly at scale, one can easily see data centers start using 4% of total electricity, and in the next decade 8%. That will start to have severe effects on the power grid, basically require planning many years in advance to setup new power plants and such.
"Moore's law and incremental improvements' are irrelevant in the face of scaling laws. Since we aren't at AGI yet, every extra bit of compute will be dumped back into scaling the models and improving performance.
yes, at least as long as you constantly develop new AI models
and you still need to run the models, and e.g. for GPT4 that is alone already non trivial (energy cost/compute wise)
through for small LLMs if they are not run too much it might be not that bad
---
Generally I would always look for ulterior motives for any "relevant" public statement Sam Altman makes. As history has shown there often seems to be some (through in that usage "ulterior" has a bit too much of a "bad"/"evil" undertone).
To cut it short he seem to be invested in some Nuclear Fusion company, which is one of the potential ways to "solve" that problem. Another potential way is to use smaller LLMs but smaller LLMs can also be potentially a way how OpenAI loses their dominant position, as there is a much smaller barrier for training them.
The size some of the large language models have is just insane. To put it into context GPT3 is a Large Language Model (LLM) but GPT4 has like (handwaveing) 5+ times as many parameters. This means the energy cost is also at least that much larger for inference.
And if we look at training instead of inference it's a quite a bit more complicated (each iteration is more costly, more parameter can also require more iterations, but at the same time you might need to spend less time in an area of increasingly diminishing returns per iteration), but if we go with GPT3 vs. GPT4 the 5x+ increase of parameters lead to a 10x+ increase in training cost (of which a huge part is energy cost, through also amortized hardware cost; ~10M$ to >100M$).
Additionally there are various analysis steps you might do when creating new models which can be hugely (energy) costly.
And that is with GPT4 and with OpenAI any major increase in version seem to come with a major increase in parameter size so with that trend we are looking at energy bills in the (potential many) hundreds of million US dollar for training alone.
Another thing wrt. inference cost is that with my limited understanding currently the best way to reach AGI and also a lot of other tasks is to run multiple models in tandem. Through this model might be domain adoptions of the same model, so not twice the training cost.
---
Domain Adaption == take a trained model and then train it a bit more to "specialize" it on a specific task, potentially adding additional knowledge etc. While you can try to do so just with prompt engineering there a larger prompt comes at a larger cost and a higher chance for unexpected failure, so by in a certain way "burning in" some additional behavior, knowledge etc. you can get nice benefit.
Joking aside: the options are to turn it off or to charge what it should cost and given that no matter what it costs and no matter what the source if it uses too much energy the better solution would be to make it use less energy rather than to look for a breakthrough energy source because that also reduces the climate impact.
[0] https://www.wsj.com/tech/ai/microsoft-targets-nuclear-to-pow...
The final proof reading and decision still is done by humans, but before that automatic feedback AI is used to find things which need to be changed during planing, certification and highlight potential problem. I.e. eliminate as many rounds of back and force between agency and plant builder and in turn save a lot of time.
Through if the final proof reading is also done by AI or relies to much on AI this is a huge issue. LLMs fundamentally to their design sometimes (in the future maybe rarely) make mistakes in forms which for humans seem arbitrary, random and absurd (in turn they tend to also not do some of the mistakes humans tend to do). The problem is even if this mistakes become rare they always can be fundamental safety issues.
If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W, which gives you almost 250 MW for just the GPUs. That sounds like a lot, until you realize that a single large power plant is measured in Gigawatts. All of Google's data centers combined are O(10 GW) which are matched with renewable power offsets [1].
Importantly, the world installed >500 Gigawatts of renewable energy in 2023 [2], mostly driven by PV Solar in China. The amount of potential solar and wind and other renewable-ish (hydro) outstrips even a 10x'ing of a lot of these numbers. But even for a single site, dams like Three Gorges are >20 GW.
There are real efficiency and scale challenges in doing AI in a single, large site. But existing power generation systems deliver plenty of power.
[1] https://www.gstatic.com/gumdrop/sustainability/google-2023-e...
[2] https://www.iea.org/reports/renewables-2023/executive-summar...
Efficiency gains should come first, long before they start looking at alternative energy sources.
> This report also analyzes prospective generation capacity in four categories — under construction, permitted, application pending, and proposed. More than 466,000 MW of new generation capacity is under development in the United States — a 13% increase over 2022. Sixty-one percent of capacity most likely to come online, permitted plants and plants that are under construction, are in solar.
China's growth in power capacity is non-trivially due to increasing demand. If the US or Europe or wherever suddenly wanted to build XXX GW per year, they could (modulo bureaucracy, which is very real).
The US or EU could likely do it, yes. If someone is willing to put up the capital to do it. The point of Sam Altman's statement is that he wants someone to do it.
> If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W
Plus ~500W for cooling
And third: renewables need to be associated with its backup like hydro/step or ... batteries which cost a lot. Gas can't be taken in as it's not CO2-free. All that unless training and inference happen when there's the corresponding wind and sun shining. And I'm not seeing that happening right now.
Maybe Apple will have some ideas with the power/performance ratio of their chips compared to what's used in graphic cards.
A lot of the 13 and 30bn parameter models are actually quite good, maybe not GPT4 but it can run on my M1 MBP without an issue, and a bit faster than GPT4 anyway.
I don't need a chat interface that is amazing at all things at once, I need it to be great at 1 subject at a time.
Why? I understand that for a normal website you always want to use the closest server possible, but for queries that take 30 seconds on average anyway, why does it matter if you're adding an extra 250ms to get it from the opposite side of the world?
If AI can legitimately replace human labor, why would anyone care if it takes an extra few milliseconds?
I want one which can instantly calculate numberwang[0] and get it right every time
[0] https://www.youtube.com/watch?v=6ofnDdi-vfw