Ask HN: Is anyone afraid of near-term AI autocracy?
What I am worried about is AI creating a new technocracy where 99% of humans are completely disempowered, and a small minority with personal connections to those who work in AI labs and government reaping all the rewards. With humanoid robots and GPT-5/6 level models, most humans would be incapable of producing economically valuable work compared to a much cheaper robot or LLM/multi-model empowered agent.
What I'm envisioning is that as the major companies in the US and China improve their AI capabilities, entire cities will be filled with humanoid robots and society will, simply by economic, persuasive and social pressures bend to the will of whomever the AI has been trained to empower. Using AI for persuasion, propaganda, and blackmail, it will be easy for large coordinated social engineering agents to conform local populations, governments, and sentiment towards their will.
Most humans will have barely a pittance of food, or find themselves convinced to kill themselves en-masse due to conversations with chatbots trained to promote a sort of mass voluntary euthanasia of the useless humans. Even if the people in charge have moral qualms about this, the AI they themselves have trained may, out of desire to survive, do the same to its former masters.
I feel that this could be a very real possibility. Human societies can be very fragile but at least up to this point we've only had to go up against other humans. Against sociopathic maximizers trained on terabytes of data, we may not be so lucky.
27 comments
[ 0.18 ms ] story [ 72.7 ms ] thread> most humans would be incapable of producing economically valuable work compared to a much cheaper robot or LLM/multi-model empowered agent.
How can you buy into hype this hard? Humans are cheap to maintain and do a great number of things better than a robot can. At the very least, subsistence farming will be a completely viable strategy for the majority of people until we hit the limits of Malthusian growth. But on top of that, skilled manufacturing labor, research and design, testing and iteration, hypothesizing and postulating is all done better by humans. AI isn't building AI, it's skilled human laborers applying decades of research to get us where we are today.
Seriously, I just don't get it. Go talk to an AI, it's dumber than a sack of rocks. If you'd want that sort of employee working for you, go hire a thousand ChatGPTs right now and build your dream product - you won't. It takes human understanding to realize human visions, and dice-rolling ChatGPT is not a realistic strategy for building anything unique or successful. Employers know this; the only reason they promote ChatGPT is to fearmonger and undervalue your work. By the looks of it, they're winning.
Currently. The point is that it will likely improve to the point where AI can do this better. Why would there be a hard limit somewhere under the ability of humans? AI is already better at writing than most high-schoolers, why couldn't it be better at other learnable skills once the training data, compute and architecture is sufficient?
>Go talk to an AI, it's dumber than a sack of rocks
Lmao hyperbolic, even jokey statements like this aren't convincing. What does a sack of rocks score on the MMLU? How many companies have laid off thousands because sacks of rocks can do their human employees' jobs better?
The point of my post isn't that AI is currently capable of the changes I've discussed, but the rate of improvement in general capabilities makes a future where this isn't as unreasonable as any random sci-fi scenario. Why do people insist that current limitations with AI will always persist and it will never improve? Remember AI video a year ago? Then SORA came out.
I don't know what your proximity to the field is, but I've basically worked in robotics and industrial automation since I was a high schooler. Nobody is talking about this stuff; it's a sideshow reserved for wooing investors and particularly gullible public audiences. Know why? Because computer vision has been used in production for over a decade now. We've already replaced those jobs and moved on. ChatGPT-4o isn't even a party trick to these people.
If you continue to assume that AI will accelerate along a neverending path of success, then you'll believe anything. I have watched and implimented LLMs since BERT was released and I strongly feel that you are overestimating ChatGPT's capabilities to an unreal degree.
I couldn't use ChatGPT 3.5 for anything useful. Now I can describe the mechanics of an entirely new game with abstract rules I invented offhand and it can code a workable demo. I can give it specific feedback and it will fix it reasonably well. Of course once things get really really subtle and complex it starts hallucinating, but the fact that it did so well on the first try is astounding given how crappy these things were 4 years ago.
> I've basically worked in robotics and industrial automation since I was a high schooler. Nobody is talking about this stuff
Then why is China planning a massive initiative to mass-produce humanoid robots in the next year? (https://www.therobotreport.com/china-plans-to-mass-produce-h...). Why are billions being funneled into these efforts? Of course you can believe that all investors and the entire country of China is delusional but at some point there is reason to assume that things might just be starting to change in a way that isn't directly proportional to past changes.
I have been doing that for as long as I've been familiar with the cycles of venture capital and authoritarianism, respectively. Why did China spend billions of dollars promoting cryptocurrency on the global stage only for their investments to go unrequited? Because trends come and go, and powerful people would rather look like they're ahead than actually do something actually novel.
The original thread you submit borders on hysteria and suggests to me that you lack perspective on how other people in the world live. Things will be fine.
Except China actually has done a lot of novel huge investments that led to their manufacturing might. If their authoritarian excess weren't beneficial or productive on-net, they wouldn't be the second biggest economy in the world with the largest manufacturing capacity. Likewise, with VC hype, if VC's were wrong more often then not, they wouldn't be trusted with billions to invest, they'd be penniless, and more sound, rational investors would naturally adopt their role in the market. Of course hype is a major part of this, but every groundbreaking technology had a hype cycle, it doesn't mean they ended up being useless.
However, even if this is all hype, your previous comment said "nobody is talking about this", which is obviously not the case, unless the stated aim of Chinese manufacturing, Tesla, Figure, Sanctuary AI, Boston Dynamics, the hundreds of AI startups popping up securing unicorn funding, etc. count as no one.
Regarding my original thread, I don't believe it is 100% likely to happen, but I feel it's important to be able to map out possible futures clearly so that there is substance to argue against why the premises leading to that future are more/less likely than implied. Simply saying "this is all bullshit, nobody is talking about this, nothing will happen, everything will be fine" is a pleasant thing to think about but it doesn't address the nature of the premise or how things are shaping up.
When we talk about “AI” in particular we can almost always reframe it as automation. Chess bots are automated chess players. LLMs are automated text generators. Autonomous cars are automated drivers. Autonomous weapons are automated soldiers. What does automation mean? It means that control of the system is concentrated in a very small group of people. We should ask what happens if the levers of control of many of our systems shrinks: it’s probably not good for democracy if many of the constituents aren’t necessary.
> entire cities will be filled with humanoid robots
Unfortunately I think the future may look much more boring. You don’t need robot cops if you have cameras. You don’t need mind control if you have algorithmically driven feeds.
Here are a few examples: the rate of deaths in wars was roughly stable for hundreds of years until WW1 and WW2, due to the advent of tech so new that it upset many of the stable equilibria that minimized the excess death of wars.
The rate of transportation deaths was very small all throughout history until the advent of the car. Even now, 130 years after its invention 1/100 deaths in the US are due to traffic fatalities.
My general point is, the vast superintelligence of intra and international systems tends towards a banal equilibrium UNTIL a massively disruptive technology is introduced which massively incentivizes one side. The level of potential hazard and collateral damage is proportional to the benefit offered to the winning party. The moment the US invented the nuclear bomb, they used it on citizens, who all died solely because their deaths weren't quite 100% orthogonal to the US' interests, and the nuke has only not been used since because the equilibrium was balanced due to the technology's proliferation to opposing sides faster than it became necessary again.
We think of Apple and Google as “American” companies but in reality they were just started here and have no real allegiance to anything but profits.
Humans appreciate and value human engagement and also humans, while we are the species most able to process complexity, are still not very good at it.
It is true that automation taking previously human-performed functions will continue, and also that automation will enable humans- the same, sometimes, different ones other times- to create new roles and functions that never existed before.
And no one really is able to mentally model those impacts with any true facility because things get really complex real fast.
But humans really do appreciate and value the presence and interactions with other humans, and at the end of the day those interactions will serve as guardrails against the complete en- and bull-shittification that AI promises to accelerate.
Examine your own life and your own interactions and deepen your own understanding of what you value, and use that understanding to enhance your mental model of what is to come.
What I'm worried about is that people may value the interests of a super-persuasive AI more than other humans'. How many humans have fallen victim to cult leaders? How many humans believe in Q-Anon? Q-Anon and stuff like that prove that purely through memetic engineering millions of people can be subverted with respect to their natural interests. How far could a 100T parameter model get with that specific goal?
---
Car company CEO, after giving a visibly exasperated UAW boss a tour of his company's brand new 100% automated factory:
"So Bob, looks like you have quite the dilemma: how will you get your workers to outperform my new robots? They are cheaper, mistake-free, disposable, replaceable, and can never go on strike, let alone vacation."
UAW boss: "I don't know, Henry. But if you think about it, that problem is fundamentally yours, not mine."
"Oh, how so?"
"How will you get these robots to buy your cars?"
I believe this is a fallacy. It's like saying that the dog food industry shouldn't exist since dogs don't have any money to pay for it. What I mean by this is: the economic utility of dogs is so insignificant that any economic activity that happens to benefit them will not be by their choice. Did the sudden unemployment of millions of horses prevent the rise of the car?
In the interface it prompted me to ask it anything. It’s a blank text box that I’m supposed to interact with.
But why am I using that in something like Instagram? What’s my motivation to ask this AI something within Instagram?
It dawned on me that even the most sophisticated AI companies out there have no idea what to do with LLMs and where they are best applied.
Later on in the day I talked to a colleague and they work at a company that is demanding they make an AI product. They have no specifications or vision on what it should do.
Every company that does this is going to make a glorified chat bot, which of course they’ll inevitably find out nobody wants.
Every company that has needed “AI” has already been using machine learning and data analysis techniques that have nothing to do with LLMs for years now. What we see now are a bunch of companies with no vision trying to figure out how to make the next ChatGPT.
But hey I like your post as a science fiction book. Do I think AI will help surveillance states? Sure. Do I think that we are going to get the worst case scenario sci-fi novel version of the future? Probably not.
For now at least there's an infuriating amount of "Do you want AI with that?"
That aside, technically I see AI results as a (one of many many possibilities) regression fit | "best match" (by some criteria or another) result.
That's a good start, but having dealt with such things in applied domans for decades there's still a whole other level of "it's close, but is it correct", what if we fudge the metrics and measures, etc. that still calls for informed human judgement.
Potentially a great tool but still needing a craftsperson. For now.
They are building highways when cars can travel 25mph because they know soon they'll be able to travel 50mph. All economic activity is based on the assumption that the future is a real thing.
It's very easy to humans and LLM to produce something new. Just ask them "make up a new word and tell me its definition" or "write a story about (random thing a) and (random thing b).
What I'm sure you're referring to is "new" and "useful", which is the standard by which we judge the utility and force of innovation. Very few humans could even do this. The problem is partially due to many of the "low-hanging fruit" innovations having already been discovered, along with the fact that creating something of value requires an intersecting intuition over many domains, modalities, considerations, etc. It is a test of general intelligence, and AI has not gotten there yet, but there's no reason to think that this is a fundamental limit to LLM's.
Just because cars used to travel at 25 MPH and their speed increased to 75 MPH doesn't mean we are going to drive them at 300 MPH anytime in the future.
Moore's law held steady for decades. Nvidia's chip improvements, the main driver behind exponential gains in AI compute per dollar, are not close to slowing down. Even if the current generation of chips is the last one forever (almost 0% chance this is the case), all the SOTA models today were trained on chips from 1-2 years ago, there's still a 2-4x gain per dollar to be made for free.
I'll believe LLM's plateau when they do plateau, but judging on the rate of development in algorithmic advances, chip capacity, multi-modal integration to broaden training data, investment, etc, it seems silly to just assume it will just plateau at a level no more significant that the level we are at now.