Ask HN: Anyone else feeling uneasy with the latest developments in AI?
I don't understand why we're not seeing a MASSIVE clamor for a pause in the further development of AI strength. Even if we don't know how to coordinate and enforce such a pause yet (at the multi-nation level), it seems we're not spending any effort in figuring how to.
I see a lot of people dismissing warnings from the frontier companies themselves as nothing but "marketing and generating hype". I myself can't call "marketing and hype" the kind of capability I see before my very eyes and interact with every day, at work and in everyday life otherwise.
Posts like the recent "An Alien Mind" by the chief scientist of OpenAI don't make me feel any better. It seems the best hope we have so far is to just hope other frontier labs will voluntarily slow down. And, regardless of the fact that alignment is nowhere near solved (if it can even be solved), it seems the current path is that we will rush toward RSI because otherwise someone else will do it first.
Does anyone feel uneasy in this way? I find it hard to think about anything else every single day. Even if ASI doesn't doom mankind, it's very unsettling to have to adapt from the world as it currently is, to "ASI now exists" in less than a year possibly. It seems life as we know it is about to change overnight, and probably not in a good way, and I fear for my loved ones, for all of the people that have no idea of what is going on and what the implications of this will be.
26 comments
[ 0.18 ms ] story [ 42.5 ms ] threadIt's like asking race car drivers to voluntarily slow down, when they are driving rocket sleds with no brakes. Simply unbridled, unprincipled megalomania of the James Bond villain variety, except that those villains had far less power at their disposal.
And no James Bond.
"worrying means you suffer twice"
Having said that, my advice to anxiety ridden family members of "just don't worry about it" isn't generally taken as useful or actionable advice, so I'll make an attempt at logic:
I think the nuclear arms race was scarier. It was literally designed as a city-destroying super-weapon. AI is at least designed as a profit-making service first, and anything it is then used for is secondary - which, to be fair, will include weapons / weaponisation, but at least it's not the primary raison d'être.
Additionally, I have a certain amount of faith in humanity's ability to deal with an existential crisis. There's an odd 'other side of the coin' to a government's desire to stay in power (which I don't usually trust) which can actually mean it will work surprisingly well to ensure it's own remaining viability in times of actual crisis. I'm hesitant to point to the COVID response, for reasons of political divisiveness of the resulting commentary, but governments generally took the advice of experts in the interest of maintaining a population to be able to govern over - which is better than the alternative.
Ultimately, I just don't worry about it. But I do pay attention to it. I worry about what I can control on a day-to-day basis, and don't really have a lot of mental space or energy for much else. I find that to be a healthy mental space to occupy.
I think my main concern is the sheer velocity of this revolution. Between yesterday and today, we’re now seeing the news that an OpenAI internal model that’s “a step-function improvement” over Astra (!) on several benchmarks has solved the Navier-Stokes Millenium Prize Problem.
At this rate, I fear governments (slow as they are) might not even see it coming before it’s far too late. I also fear that AI researchers, despite being well-intentioned, might turn reckless under the pressures of the frontier labs that employ them to accelerate at all costs.
But following your logic, I agree, it’s probably not hopeless yet. And I do believe there are highly smart people with more power and influence than me that are keeping a close eye on this as well, and they won’t stand there with arms crossed if they believe doom is impending (so I hope).
Good reminder about worrying about what we can control, and thanks for sharing your perspective on this topic.
People seem to fear that AI will somehow fuse the perfect truth of computation with the self-improving aspect of evolution. Most of these people are content to ignore the evidence of the physical world, and that evidence says that, basically, we're doing the best we can. Nature has not made a mistake in creating individuals. Individuals seek to make a mark on the world and have a will to survive. Individuals have not been mistaken to form societies and institutions. Societies and institutions have not been mistaken to band together into nations which compete fiercely, fight wars even. Living in a pluralist world means contradictions abound, and thrive and grow in spite of and because of each other.
Models aren't even sort of the same kind of thing. They predict averages. They smooth over differences and deindividualize. They can't justify their own survival. They aren't energy efficient enough. They need humans because we're the only thing that can create value on the scale of an entire society, and using LLMs has little it nothing with our ability to do that
Instead of USA vs Russia, it is now USA vs PRC. SO if either slows down, the other will win by default. So no one is going to slow down.
This comment was written 100% without any AI.
This might be my own fault, but I never read/watched/consumed too much sci-fi, and the kinds of sci-fi I did consume were mostly dystopian works or cautionary tales. I just have a hard time in general grasping how can we hope to influence and control beings that are orders of magnitude more intelligent than ourselves. Recent events don’t give me a lot of hope that frontier labs have a good grasp on this either (and by their own admission too).
But I do think we are not anywhere close to AI competency for fine motor skills, e.g. microinjection to make transgenic animals (https://pmc.ncbi.nlm.nih.gov/articles/PMC8361322/figure/fig2...)at scale is still really difficult.
It is also unlikely that AI will start a skynet-like scenario as those nuclear launch systems are air gapped and require multiple authorizations.
I’m assuming you’re implying that ASI will solve death. That seems to me like a possibility; just as the possibility that ASI will kill us all faster rather than solving death. I also don’t know what life will look like post-ASI, and whether it would be an enjoyable and fulfilling existence.
I’m curious about how is it that you’re fully sure that ASI will be a positive thing; it’s not at all obvious to me.
It's either AI or I get very sick and die, so my money is (literally) on AI, and the faster the better.
I fear for my loved ones when the bills come due and we don’t have an ROI on it, and any model that’s made is quickly absorbed into cheaper models. That the utter hucksters who have given takes ranging from massive exaggeration to willful Big Lie will somehow tell the truth while they stand to gain untold billions.
I’m not sure if it would be worse if they fail utterly or succeed completely, but my bet is we somehow manage to end somewhere in the middle, towards the former. There’s been no accounting yet for the lies told to get the money.
I really wish I could see the potential of those technologies, but that potential is put to the service of humanity's worst.
It is not an intelligence if it cannot detect bullshit during its "training".
That being said, companies have to make their money back on this eventually and they're already jacking up their prices, even if we get to AGI I'm not sure most people will be able to afford to use it to handle everything in their lives, at least not for quite a while (and companies won't be able to afford to replace every human with an AI counterpart).
It's already getting expensive to go too heavy with something like Claude, to the point where my company hosts internal seminars on how to reduce token count (for the company's employees) and also runs out of tokens before the end of the month for the company every month, and we're going pretty heavy on A.I. usage within the company.
I found out the monthly usage they try to keep everyone at, if possible, is around $200, although I also found out the other day that there are people in the company using as much as $1500 in a month, although they're also highly involved in a bunch of projects at the company.
And I highly doubt companies are going to just leave servers running AGI models going just so it can do some sort of hostile takeover. It's still ultimately reliant on electricity and massive compute power.
Truly, even with the current level of AI, we should be concerned how people should learn. This is true of adults, but incredibly important for children. Should we become dependent on these tools to the point where we feel like we can't achieve our day-to-day work?