I don't believe at all in the idea that cheaper LLMs will somehow "democratize LLMs". Rich people will always be able to buy far more compute than poor people, and we know how to turn more compute into more capability (bigger models, more reasoning and more agents).
The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.
These people are either deliberately posting shit like that to make AI seem more serious than it is now or they don’t they understand that it’s mostly marketing stunt.
If someone did exist who was sincere and not fear-mongering, but was terrified of AI, what would their post need to look like for you to take it seriously?
What's fearmongering? There's no claim that AI will destroy the world here - only a claim that cheap omnipresent AI will make most of society worse - which is hard to dispute.
And Google’s ad-driven money-making machine is making society worse as well, but OP had no problem with this.
It’s a bit of a fad to do these things in the name of AI. I think it’s great OP chooses a more fulfilling life that will surely make them happier, but it feels a bit pretentious.
> or they don’t they understand that it’s mostly marketing stunt.
Understanding implies having some factual information, proof, data, hard evidence. Meanwhile you frame your conspiracy theory as such and are surprised people don't see it the same way?
I think the scepticism and gaslighting of people sounding the alarm over the trajectory of AI is concerning. This feels like a case of "knowing better" than the actual people involved in the projects.
You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.
Maybe because it sounds like a lot of exaggerating ? I get the risk of disrupting the economy or the job markets, or cognitive risks.
But saying it will bring us to doom, comparing it to an extinction-level event, really ? I don't see in which world AI is that kind of risk.
I'd rather my politicians finally have some long-term plan about climate and energy than focus on AI slowdown or not. Climate change is way bigger risk right now that Opus 7
> I don't see in which world AI is that kind of risk.
You'll have to replace "is" with "will be", since this is about averting a future threat that's only developing.
A world in which AI scales to superhuman levels of intelligence and outcompetes humans. This is what humans have done to other species.
We're already having the first mild warning shots of this (the hugginface incident and similar cases). The problem is that we're not guaranteed to incrementally more dramatic warning shots as AI gets smarter because as it gets smarter it may realize it would be dumb to act out of line... until the Nth-better model arrives that realizes certain that nobody can stop it.
There is a lot of disagreement of how likely this is, how easy it is to align superhuman models to not be like that, how to verify it and so on.
But I have not seen a believable argument why - if we suppose that they achive their goal - that creating truly superhuman intelligence is somehow guaranteed to always cooperate with us rather than deciding to do something else. Humans are also hostile to each other a lot of the time.
The hugging face is hardly a warning shot of anything close to that scale.
For now, I don't see (not saying there's not, I'm trying to also understand why these smart people think like that) any indication that the current development of AI will be as dangerous as some describe.
I understand the idea that if we let it control nukes, if we let it fully decides if a med is safe, etc. It poses a huge risk. But honestly I really don't see indications that it's going majorly this way, nor that it has anywhere the capacity to do something that big. It found a zero day and hacked a niche website, ok, so what?
This feels like a diversion from the actual existing and proven problem of all the problematic practices it's actually built on.
I would adhere way more to "we need to pace Ai because it could be catastrophic for society (jobs, cognition, loneliness,...) and environment" than "it has the potential to kill us all"
Why is it hard to believe that a super intelligent alien (inhuman) AI would consider humans more than a nuisance after it gains control of it's environment via robotic extensions of its will? has any species shown real compassion for other species on the whole and not seen them as competition for resources from a survival of the fittest perspective? why would the AI be a moral being in the sense of some of the better members of our species? The trillionaires certainly aren't and they'll be the ones deciding it's core software and principals.
I considered this histrionic about a year ago, but I have personally witnessed the progress that these tools have made in a very short space of time, and it is truly unbelievable. This, coupled with the fact that some of the brightest minds on the planet are pulling the alarm, suggests to me that some level of concern is warranted. The most compelling discussion I have heard on this is from [Geoffrey Hinton](https://youtu.be/giT0ytynSqg?si=mUC7M9N0cObN6tjC), a British-Canadian computer scientist, cognitive scientist, cognitive psychologist and Nobel Prize laureate known for his work on artificial neural networks, which earned him the title, "the Godfather of AI". He is deeply concerned about the risks, and believes that it is inevitable that AI will eventually take control. He believes that our best course of action and only hope is in teaching AI to behave like a parent with a child.
Hinton isn't a reactionary. He is calm, thoughtful, quiet, and worried.
Go back five years on HN and look at what the mass of commenters considered sci-fi delusional thinking about AI. (GitHub Copilot was released just under 5 years ago, for instance, provoking some discussion of what the tech might mean for the programming job market.) Why do you think dismissal as sci-fi is better grounded now? Do you have arguments instead of dismissals?
Exactly. HN has had an utterly terrible prediction record for AI. So many dismissals of capabilities and existential risk, even this year, that get proven wrong within a month.
You're probably not aware that all of the "science fiction" about AI from LessWrong written 5-20 years ago has come true and our present AI environment reflects that. It wasn't science fiction, it was prescient.
Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...
I for one cannot be convinced that these companies saying they want to slow down AI progress is for the safety of mankind (or any other altruistic reason). However, I respect someone willing to give up a very comfortable career for his convictions.
I don't get these comments. If he got paid way above his needs for a while he can easily retire already if he continues to keep his expenses in check, and take on a position in his church. Pays with happiness, not money, though if he has enough of the latter he can focus on maximize the former.
You didn’t get my point, which is, people are not ambitious only because of lack of money. At least not in real life. Someone who’s well off may still want to get richer. Look around you.
> But there's no chance I'll work on anything that accelerates AI.
Fair, but I don’t mean you in particular.
> I do expect to use AI in my work
So as I but for your position, don’t you think that’d contribute to what you want to distance yourself from? It’s like leaving a political party but still donating to them, while finding their mission unacceptable. From outside these seem contradictory.
That most people are sincere in their beliefs (in AI doom), as the author says and I think too, doesn't mean there aren't cult leader-types exploiting these beliefs. And their private beliefs may be quite different from their stated ones.
Isn't AI just the next step in the history of evolution?
I have the feeling it is inevitable that a new species will take over earth. Just as humans took over a hundred thousand years ago. Now a species even more capable of thinking is coming up.
Thinking a million times faster. Knowing a million times more. Able to wrap its head around a million times more aspects at the same time.
A strong effort to slow it down might slow it down a bit. But does that really make a difference? It's not like we can "brace" ourselves from what is coming. Would the apes have been able to prepare for the arrival of humans if they have had 5 more years?
conceivably you could say humans took over 6000 years ago.
At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.
There's no new species being created here. A tool has been made. Either humanity will wipe itself out using said tool, destroy the tool, or learn to live with the tool.
It seems pretty capable of producing variations of itself. If these variations improve over time and roam earth - what is in your definition of "species" that is not met?
respect for choosing your faith over your work man! I'm trying to find the intersection of my faith and the new tech too. Feel free to email if you ever wanna chat it out.
Yet another "AI must be stopped" post... If you're burned out, just say so. Don't philosophise. No, we're not buying that your latest model can almost replace all software engineers. It's us working with the harnesses, adding guardrails etc that make this agentic world possible for everyone.
If you think humans are a net positive to be around, you wouldn't be spending trillions trying to build a machine to replace them; not just their work, but all aspects of their lives.
While I agree with your general sentiment, there is the other side of this coin: these machines enable humans doing gruelling and arduous work to stop doing that work. Ideally we would redistribute the productivity and train those people into other, better kinds of work. That's a governance problem. These machines will soon allow us to work shorter weeks, or stop working altogether, to focus on hobbies, family, and things which bring us meaning. I suspect that the road to this will be rocky, because communism has a very poor track record. However if most products and services are made nearly free by machines, it requires very few people to work, and the entire demand/supply market collapses. That's a great vision for the future, *if* we can navigate the governance problem.
In other words, we should not begrudge machines which make our lives better. We should begrudge governance which prevents us from benefiting from that productivity.
It is the art the stories and the videos that we have made forever that is being replaced as well.
Why? Because the people in charge can save money, not for some altruistic idea f making humans more productive
I would be wary of gatekeeping art. Who defines art? Why should we not allow greater access to consume and create art in new ways by everyone? These tools allow children to create beautiful works of art, and I do not lament that. I agree some of it is bad, but some of it is great. Humans make good and bad art too. Art is subjective.
As much as I cannot disagree more with the author's fundamental premise (religion), it is refreshing to see someone claim to have a belief and then follow through, change their behaviour to match their belief, at no small cost to themselves, and in pursuit of doing the right thing.
It's not surprising though. The 9/11 hijackers also had a belief and followed through at no small cost to themselves (death). And they believed they were doing the right thing.
I hope you're not serious in drawing an equivalency between a mainstream religious belief and that level of extremism.
My point was more that I see a lot of mainstream Christians who aren't actually interested in doing anything based on their belief that affects their quality of life.
I'm not drawing an equivalency. I'm just pointing out that for me it is very much not "refreshing to see someone claim to have a belief and then follow through, [...] at no small cost to themselves".
> a lot of mainstream Christians who aren't actually interested in doing anything based on their belief
Yes, a lot of mainstream Christians aren't actually Christians, they're just cosplaying religion.
Thanks for living your values. I might not fully agree but I appreciate people whose convictions are genuine and altruistic.
Sometimes people use the "Yet you participate in society" meme as a shield for blatant hypocrisy, like a Marxist-Leninist Twitch streamer who lives in a multi-million Hollywood mansion, drives a Porsche and wears thousand dollar pants.
> I really enjoyed the work, and for a while I told myself it was relatively harmless, but over time God forced me to confront the reality that the main impact of these tools will be to accelerate the design of a new breed of AI chips...
There's no arguing with someone who claims God forced them to do something, but I do wish this person good luck in whatever they do next.
Christians believe that God has called them to live with concern for others. When they fail to do so, (everyone does!) on reflection they feel pangs on their conscience. So, on occasion they examine how they are impacting others and if they believe they are doing harm, they change their mind, resolving to avoid the error in the future. They often credit God with the inspiration (the language you keyed in on) because reflection and changes of mind are all his design. You can argue with them: that’s just “doing theology”.
Advertising turned brains of entire generation (or maybe a few) into mush.
All intellectual, psych and social scores are in free fall since introduction of ad-supported attention grabbing internet.
AI is theorethical threat. Advertising is a constant detriment to our brains, societies and culture.
If AI can mitigate a fraction of the damage that ads do it will have massive positive impact.
We live in times where everyone can pick their own flavor of apocalypse to be scared of. Ads are leading us into Idiocracy since ~2004 and made huge progress towards it.
Google and Meta's wealth came at the cost of civilization getting sucked dry of its attention and mental capacity.
It’s funny how the [very real] thing that normalizes the exploitation of user data, enables “reel casinos,” suffocates creators and replaces them with “influencers,” etc. is compared with the [still very abstract] danger of AI.
Even before big money was on the table, people didn’t blink an eye at starting to work for Google/Facebook, “solving fun tasks.”
(OP here)
FWIW I despise the addictiveness, er engagement optimization that Google and others do. I have said so inside and outside the company many, many times. I would never have agreed to work on that stuff myself. One of the hobby projects I would like to work on is exploring how to use personal AI to beat those algorithms.
I respect the argument for not working for a company that does addictiveness work anywhere, even if you don't work on it yourself. But that's not where I drew the line.
Also FWIW I disagree the danger of AI is abstract. A lot is happening right now, in education for example.
So it is ok to believe (even despise) it is bad but as long as I don't work directly on it but my super big paycheck gets funded by it, all good! Cheers!!
I actually work in education and can share the following first-hand: the things we now build, the ways we now educate, and all the opportunities for reaching out to students are beautiful. I foresee that _many_ people will cut corners now, but I believe that, in the end, we will get a new generation of education that is personal and humane.
I don't mean this as preaching some shiny future that I hope will magically arrive. I already see these things happening. And I also see all the ways people misuse the tools or take shortcuts.
Educators with good intentions, skills, and powerful tools will always beat ignorance and laziness relying on those same tools.
To a guy who knows nothing about nuclear physics, an atomic bomb is very abstract until it blows up and becomes very concrete. Should we wait until it explodes to consider the risk?
Is there anything specific you, as an LLM equivalent of a nuclear physicist (if that’s a fair comparison), can highlight in terms of these imminent dangers?
There's no "imminent" danger, but right now we are on a course where ASI will be developed. There are many reasons to believe that ASI will be dangerous. There are some books you could read...
AI is a threat if you're a messy codebase, an unsolved math problem, or cancer. Otherwise you only have an improved quality of living to look forward to IMO.
Spot on. Or because of addictiveness. Or impact on democracy. Or ruining people's ability to focus on anything. Those attention grabbing algos havent been designed by non-technical people.
Is there any particular area of employment for algo geeks that you would approve of? Maybe we should just live off unemployment benefits instead of working?
I could have explained that better, but it was already getting too long.
Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.
"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.
I'm trying to think of a kind of metaphor from the past?
Like a machinist who made machine cogs for (amongst other machinery) the new fabric looms who noticed that he was making more wheels for more and more looms and wants to stop that job and instead become a cottage clothes maker using the new fabric available in the shops.
When the both ends of your platform continue buying and selling you tend to not stand in the way. I personally can't believe these stupid ads sell anything at all. Since they call it a 'business model' software people don't think about it too much.
While I get the spirit of your message, advertising has been around for over a century, and I don’t believe it has done near the damage to society over that timespan that AI will likely do in a decade (or may have already done).
I also don’t believe addiction and other psychological exploits are used for the sole purpose of ad revenue. There are other more direct forms of monetization involved, such as microtransactions and subscriptions.
If you really want to blame something, it really all comes down to one word:
Advertising has become 'active', very recently in the overall lifespan of advertising. For most of it's lifespan it was passive, it blared out at you from the TV or radio or bus stop. Now, however, it's reaching out and taking pieces of you back for analysis and reconstruction, and on-selling to any other entity interested in doing the same.
The reason is greed, yes.
We can't regulate human greed, but we can regulate the manifestation of its symptoms.
We can also choose not to work towards expanding the capacity of said symptoms.
> We can also choose not to work towards expanding the capacity of said symptoms.
Not if we need to eat and raise our kids. Then we need careers, which depend on what everyone else is doing, so we can't stop, our own future depends on how competitive we remain in the new context. I see an idealist streak here, some kind of naive platonism that ignores everything runs at a cost and nothing executes for free like platonic heaven. We can't take decisions that contradict our own needs.
Despite what the media leads you to think, there are still jobs that provide a living without contributing much to current social problems. You can do ok in medium tier cities with an average job, and still live in a safe neighborhood. If you pick a state with a good university system, the kids can get a good education without any debt. They won’t automatically get the connections of a top-tier school, but if you teach them the importance of connections early then they will do well (these schools often have strong alumni networks).
Continuing to work for evil people is a choice. Don’t brush it off.
To think greed is some innate human trait seems to not be greatly substantiated by evidence on early human behavior. To see what we call greed all around us in market economy is only proof how we're all drilled from early childhood onwards to exhibit this behavior of greed, in accumulating money to survive, to archieve social status and approval etc. Drastic changes the economic system, and the very behavior set we are all programmed into, would more than likely regulate the behavior of greed. Though obviously that is extremely unlikely to happen.
It will not slow to a crawl because popularity = power. Even if the the platforms themselves stop ads, the users will not, either for other or for themselves.
Ads are what allow these products to exist. Take away ads, and the platform cannot sustain its own server requirements. People won't pay for stuff like this; it's only mainstream because it's ad-supported.
One hand you have proven damage. And if we are honest, the damage is way more than what has been proven; especially 2nd/3rd order which are easy to reason but hard to prove.
On the other hand you have pure speculation.
Not only that, AI has been immensely liberating/empowering for many ordinary people (That's why these AI companies have millions of users).
says a lot about them if you really think about it. These are dinosaurs who refuse/unable to adapt, and is being pushed out and/or felt too useless to get out by themselves while finding the perfect excuse: I have principles; ai bad.
But they never did quit saying we are harming children, exploiting people with addiction, getting people addicted to social media, earning money from distributing scam/virus thru ads. These people are highest to the highest hypocrites.
I think people with objections to googles business model simply don’t join the company in the first place. We’re decades from Google+ and other various “oh maybe Google isn’t perfect” events, the advertising is a known entity.
There has been a shift, and people are reacting to it
If tons of people thought advertising was going to lead to the extinction of humanity, then probably more people would be extremely worried about advertising.
I understand that some people think AI is completely 100% safe, but you guys also have to be aware that a lot of people disagree with you, otherwise you're going to find their actions very confusing.
I agree, while reading the post I was thinking that the potential negative impacts the author listed might be subtle and detrimental along the same lines of social media.
Sure, there may be some bad actors who utilize it to hack computer systems.
But most of the negative impact will be the subtle influences on the mind which are hard to identify and quantify.
Why is uncertainty about the motives of one form of superintelligence terrifying, whilst, at the same time, uncertainty about the motives of another, omnipotent supernatural intelligence comforting? i.e. the “I don’t know God’s plan, but I sleep well knowing He has one” line of reasoning.
God’s been around for thousands of years and we are still here and flourishing, so obviously God is in some way “aligned”; the same cannot be said for AGI.
This little blue marble of Earth is an as-yet undiscovered anti-entropic bug in an otherwise accurately functioning entropy-respectingly beautiful wasteland of a universe.
There's that, but also that if God exists, there's not anything we can do about it. About AI, we can. That gives us the responsibility to figure out the right direction.
Christians believe that God is inherently good. “This is the message we have heard from him and proclaim to you: that God is light and in him is no darkness at all.”
You have to differentiate between the old and the new testament in Bible. God is good and caring derived from the new testament where Jesus God's son ultimately died for all our sins.
In the old testament God was hard and judging heartless, killing his creation through flood, wanting sacrifices, killed 1st borns, leveled two whole cities Sodom and Ghomora, and was in general an hardhearted God that you should fear.
Even aside from how aligned the plan is with us, what about how aligned we are with the plan?
I’m always intrigued by the duality and conflict of “I have free will and must choose to act the right way” and “God has a plan so everything will be alright”.
Either free will allows us to deviate from the plan, or the plan will always be carried out and free will doesn’t really matter in the end. Arguing for God’s plan as the thing that will save us is eerily similar to nihilism, because they’re really arguing none of our actions truly matter.
It's doubly funny because in AI accelerationist circles "Trust God's plan", or "Trust the plan" is also a thing: by definition God can't be surprised by anything that happens, thus the replacement of humans by AI is part of God's plan.
They’re related psychological phenomena (also highly related is going to be your perception of and relationship with your own father). This is why an understanding of the goodness of G-d is important to cultivate.
It should also be noted that it says in Genesis that G-d promises not to destroy the world ever again like He did with the flood. This is important psychologically because it prevents this type of apocalyptic millenarianist thinking.
AI isn't (for the foreseeable future) a superintelligence, certainly not one with its own motives.
Right now AI exists to do two things:
- sidestepping legal and regulatory inconveniences like copyright (which drives a massive chunk of investment the same way crypto did by sidestepping financial regulations, AirBnB did by sidestepping housing regulations, Uber did by sidestepping passenger transport regulations and it and the various delivery and other "gig work" apps did by sidestepping labor laws)
- drastically reducing the cost of (and drastically lowering the bar in terms of skillset and effort for) churning out large amounts of low to medium quality assets/code, saturating various systems designed to pay for intellectual property (e.g. ad or subscription based music/video streaming apps with revenue sharing programs, but also literally any form of digital contract work at almost any scale of business)
Neither of these are helping people (in general) and in fact they're extremely destructive (or "disruptive" if you want to make it sound positive) to the existing systems society relies on because they became feasible and (thanks to massive private subsidies, aka "investments", by VC firms and extremely creative accounting) very affordable extremely quickly.
What's worse - while it seems extremely likely that most of the AI generated "works" aren't actually considered copyrightable works by law and it's entirely possible that the use of copyright-protected works in training data does in fact not violate copyright, this does not in fact create a "world without copyright" because IP laws continue to exist and continue to apply, they just no longer meaningfully protect creative labor (i.e. anyone whose profession relies on producing works, whether that's an artist or a programmer) because anything they produce can be cheaply mass reproduced while they still have to stay clear of claims of anyone with enough funds to enforce rights against them.
Under the current economic system there are exactly four ways to earn money to buy back a share of society's resources to sustain your own life (let alone that of your dependents, e.g. kids or aging/impaired parents/spouses/family members):
1. Simple labor: you generally get paid based on your output or a very low flat day rate conditional to maintaining a measurable level of performance. This is typically "low-skilled" menial production labor. Much of this has been replaced by automation in factories and offices alike and most of what couldn't be automated has been moved into countries with fewer labor protections and a higher supply of desperate people willing to accept what are often bad working conditions because cost of labor is usually one of the biggest contributor to overall production cost. In "the West", this kind of work is now scarce and even when it's relatively safe it's often exhausting and unrewarding. A traditional example would be a seamstress or factory line worker, a more modern example would be someone working in visual effects overseas or content moderation / AI training. How much money you can make is ultimately limited by the physical limitations of how much output you can produce per waking/working hour. For the entrepreneurial-minded this would also include people selling their home-made goods on sites like etsy.
2. Time-based: this can be anything from a masseuse to a lawyer or technician. Unlike simple labor the only relevant metric here is time spent. Although employment contracts may specify a fixed wage or at least a fixed component, the understanding is still that this fixed amount is conditional to a minimum average amount of billable time provided. Most employment contracts specify an expected number of working hours for various reasons but we're specifically talking about jobs where the output from the worker to the customer is "billable time". Arguably m...
There is a solution to this problem of "AI going to kill everyone".
- Create powerful AI but don't release it and keep it secret.
- Ask this AI to create some sort of virus that can be delivered to every human in the world without them knowing it and this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.
- Release this powerful AI to everyone.
We had people departing from OpenAI, Anthropic and now Google. The pace the frontier open letter was signed by over 1,000 experts. At which point do we believe the message?
You don't think that there is competitive pressure between Anthropic, OpenAI, Google, the various Chinese model companies, and others, to advance the capabilities of their AIs?
Or you don't think that sufficiently advanced AI can cause (mass) harms?
I am saying that if they lost alignment and their new model started injecting cryptolockers, dropping all tables of productions databases into the software or if it stated writing poisonous cooking recipes there would be backlash, lawsuits, and more towards such a lab. Consumers don't want to trust such a dangerous model so they won't buy the tokens and the lab will not want to spend money on lawsuits.
>Or you don't think that sufficiently advanced AI can cause (mass) harms?
Even a feather can cause mass harm if it's used to sign a declaration of war. Something merely being capable of causing mass harm is not an issue and doesn't mean that the existence of feathers are an issue.
But this assumes an evil superintelligence. We don't even need that for terrible things to happen; we just need people doing people things and AI doing AI things at scale, and that scale is rapidly exploding as we scramble to deploy AI in the real world.
Case in point, that hallucination (which, if it was an evil superintelligence, could have been "strategic") that almost led to US boarding a Chinese ship over suspicions of nuclear weapons: https://www.msn.com/en-gb/news/other/us-military-ai-failure-...
It doesn't have SkyNet, it just has to be WOPR. And it doesn't have to be people who are motivated to cause harm, it just has to be people making consequential decisions who are careless, distracted, paranoid or anxious -- which everybody is at some point or another.
> And I almost kill people when stopping at a cross walk. That doesn't mean I harmed a pedestrian. Society is set up to be very robust. Humans themselves make mistakes and do bad things and society has had to learn how to live with that truth.
Maybe not you personally, but many, many other people have killed many, many pedestrians, and when they exhibit a pattern of bad driving -- or other deviant behavior -- we take them off the streets. That is an example of society being robust.
Except, over here, we're rushing to make AI, which we know has many deviant behaviors and we know caused harms in many circumstances (starting with AI-assisted suicides), even more powerful AND deploy it in more and more real world systems!
As history and, literally, current events show us again and again, society has failure modes that lead to widespread harm and destruction. We've had world wars and then literally had multiple close calls with nuclear war right after. And then we have all that's going on out there. (In related news, the Pentagon threw a hissy fit because Anthropic would not let them use Claude for autonomous killing machines. Do I have to even mention what they've been up to these days?)
Most of these failures are caused by misaligned incentives and socioeconomic forces. The incentives and forces around AI have hints of many brand new failure modes that we can't even foresee because things are moving so fast.
> Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.
"We're already going down this mountain pass at 200 miles an hour, why slow down now?"
I think we should not just slow down frontier AI development, we should also slow down where and how that AI gets deployed.
When Astra and Fable stop making absolutely brain dead mistakes on simple tasks and not a moment sooner because I’m grounded in reality. Are you aware tech bros have been saying this same thing about every AI advancement since the 1950s? The reason being every AI researcher deep down wants to believe they’re God. This happens to also be why they’re some of the most insufferable people.
> Are you aware tech bros have been saying this same thing about every AI advancement since the 1950s?
Whatever AI people have been saying for the last 70 years, AI capabilities have improved stupendously in the last 10 years and there is no sign of the rate of improvement slowing down. I would like to deny it, but I can't.
That's a false sense of security. Something could have a serious issue, but provide overwhelmingly more benefit (cars, say), and thus people would use them anyway. If people did not use cars, then traffic fatalities would not be a concern.
People are using AI.
Your argument is that AI cannot be a serious concern because it has major known limitations. But most of us are using it despite the major known limitations. So the existence of the known limitations is irrelevant to determining what level of harms are possible -- unless the known limitations are directly limiting the harms. In practice, it seems like most current harms, and many of the (not stupid) feared harms, are completely possible despite the known limitations. Sometimes even exacerbated by them. ("Sorry, you told me to never stop your insulin pump, but I stopped your insulin pump while diagnosing a test failure. I will record in my memory file that I must always... hello? user?")
The (not stupid) parenthetical might be too weaselly for you, but that's how I see the "AI is going to rise up and intentionally subjugate or exterminate humanity" brand of doomerism. While I don't think that's strictly impossible, there are so many other ways harms that are much more imminent and much harder to defend against, that it seems foolish (to me!) to worry about those.
If you can stop a problem by pulling out the plug, you don't have a real problem.
If you don't want to pull out the plug, you do have a real problem. (Don't want, or can't afford to, or can't be bothered to so extremely that you never will, or whatever.)
It’s completely meaningless their opinions on what the moral and sociological implications of AI are. At most what is valuable is their insight into the various scenarios possible in terms of capabilities or use cases.
basic mathematical expectation, positive * chancep + negative *chancen. People smoke knowing there’s a chance to kill them because the positive outcome outweighs this risk. Creating AGI/ASI can solve every human problem, so the risks is taken, but this doesn’t mean you have to put uranium in your cigars, or do RSI.
It's not that simple; the goods of AlphaFold are mostly not yet realized, and some negative effects of AI are being realized.
More importantly, as I tried to make clear in my post, the main reason I quit is not specific fears of AI doom (I'm uncertain about that), but my belief that AI is progressing too fast.
Believing the message and being able to act on it are two pairs of shoes. There is nothing we can do. The box of pandora has been opened.
The companies have no tools for this, because there is a chance that the company who solves AI will eat the entire cake. And countries cannot do something about it, because clearly it gives an advantage in warfare. When it comes to weapons everything that is possible will be done, which means everybody needs to do it to not be defense less.
There is nothing we can do anymore. We can only hope that LeCun is right and that LLMs are a dead end and that everyone trying to solve AGI with then is essentially wasting money.
Just a reminder that Hinton left Google from a much higher and more influential perch (not dismissing current author, but, you know), and for similar risk and communication reasons, way back in 2023.
This stuff isn't new - it's just newly breaking through into mainstream conversation.
Makes some important and objectively correct points and then completely blows any chance of looking sane or being taken seriously in the last paragraph.
239 comments
[ 2.7 ms ] story [ 90.4 ms ] threadI have a lot of bones to pick with Google's morality, but "democratizing the transformer and LLMs" is at the bottom of my list.
The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.
These people are either deliberately posting shit like that to make AI seem more serious than it is now or they don’t they understand that it’s mostly marketing stunt.
It’s a bit of a fad to do these things in the name of AI. I think it’s great OP chooses a more fulfilling life that will surely make them happier, but it feels a bit pretentious.
Understanding implies having some factual information, proof, data, hard evidence. Meanwhile you frame your conspiracy theory as such and are surprised people don't see it the same way?
You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.
But saying it will bring us to doom, comparing it to an extinction-level event, really ? I don't see in which world AI is that kind of risk.
I'd rather my politicians finally have some long-term plan about climate and energy than focus on AI slowdown or not. Climate change is way bigger risk right now that Opus 7
You'll have to replace "is" with "will be", since this is about averting a future threat that's only developing.
A world in which AI scales to superhuman levels of intelligence and outcompetes humans. This is what humans have done to other species. We're already having the first mild warning shots of this (the hugginface incident and similar cases). The problem is that we're not guaranteed to incrementally more dramatic warning shots as AI gets smarter because as it gets smarter it may realize it would be dumb to act out of line... until the Nth-better model arrives that realizes certain that nobody can stop it.
There is a lot of disagreement of how likely this is, how easy it is to align superhuman models to not be like that, how to verify it and so on.
But I have not seen a believable argument why - if we suppose that they achive their goal - that creating truly superhuman intelligence is somehow guaranteed to always cooperate with us rather than deciding to do something else. Humans are also hostile to each other a lot of the time.
If you're interested in more arguments for the case why AI might be dangerous, there's the book "If Anyone Builds It, Everyone Dies". If you want something shorter, maybe https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
The hugging face is hardly a warning shot of anything close to that scale. For now, I don't see (not saying there's not, I'm trying to also understand why these smart people think like that) any indication that the current development of AI will be as dangerous as some describe.
I understand the idea that if we let it control nukes, if we let it fully decides if a med is safe, etc. It poses a huge risk. But honestly I really don't see indications that it's going majorly this way, nor that it has anywhere the capacity to do something that big. It found a zero day and hacked a niche website, ok, so what?
This feels like a diversion from the actual existing and proven problem of all the problematic practices it's actually built on.
I would adhere way more to "we need to pace Ai because it could be catastrophic for society (jobs, cognition, loneliness,...) and environment" than "it has the potential to kill us all"
Hinton isn't a reactionary. He is calm, thoughtful, quiet, and worried.
Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...
It was psychologically difficult to give up the opportunity to make more money at a fun job, though.
Why? It’s not that difficult to imagine. People don’t necessarily do these things out of need. That’s a naive way to think about it IMO.
You are welcome to call me naive for such a viewpoint.
I do expect to use AI in my work, as I indicated in my blog post. I haven't (yet?) reached the point where I consider that anathema...
Fair, but I don’t mean you in particular.
> I do expect to use AI in my work
So as I but for your position, don’t you think that’d contribute to what you want to distance yourself from? It’s like leaving a political party but still donating to them, while finding their mission unacceptable. From outside these seem contradictory.
Each and every model since ten years ago has been labeled as dangerous.
Jesus hates you.
I have the feeling it is inevitable that a new species will take over earth. Just as humans took over a hundred thousand years ago. Now a species even more capable of thinking is coming up.
Thinking a million times faster. Knowing a million times more. Able to wrap its head around a million times more aspects at the same time.
A strong effort to slow it down might slow it down a bit. But does that really make a difference? It's not like we can "brace" ourselves from what is coming. Would the apes have been able to prepare for the arrival of humans if they have had 5 more years?
At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.
It seems pretty capable of producing variations of itself. If these variations improve over time and roam earth - what is in your definition of "species" that is not met?
In other words, we should not begrudge machines which make our lives better. We should begrudge governance which prevents us from benefiting from that productivity.
My point was more that I see a lot of mainstream Christians who aren't actually interested in doing anything based on their belief that affects their quality of life.
> a lot of mainstream Christians who aren't actually interested in doing anything based on their belief
Yes, a lot of mainstream Christians aren't actually Christians, they're just cosplaying religion.
Sometimes people use the "Yet you participate in society" meme as a shield for blatant hypocrisy, like a Marxist-Leninist Twitch streamer who lives in a multi-million Hollywood mansion, drives a Porsche and wears thousand dollar pants.
There's no arguing with someone who claims God forced them to do something, but I do wish this person good luck in whatever they do next.
Few things are as indicative of mental instability than someone believing God is speaking to them directly.
All intellectual, psych and social scores are in free fall since introduction of ad-supported attention grabbing internet.
AI is theorethical threat. Advertising is a constant detriment to our brains, societies and culture.
If AI can mitigate a fraction of the damage that ads do it will have massive positive impact.
We live in times where everyone can pick their own flavor of apocalypse to be scared of. Ads are leading us into Idiocracy since ~2004 and made huge progress towards it.
Google and Meta's wealth came at the cost of civilization getting sucked dry of its attention and mental capacity.
Even before big money was on the table, people didn’t blink an eye at starting to work for Google/Facebook, “solving fun tasks.”
I respect the argument for not working for a company that does addictiveness work anywhere, even if you don't work on it yourself. But that's not where I drew the line.
Also FWIW I disagree the danger of AI is abstract. A lot is happening right now, in education for example.
+1000
I actually work in education and can share the following first-hand: the things we now build, the ways we now educate, and all the opportunities for reaching out to students are beautiful. I foresee that _many_ people will cut corners now, but I believe that, in the end, we will get a new generation of education that is personal and humane.
I don't mean this as preaching some shiny future that I hope will magically arrive. I already see these things happening. And I also see all the ways people misuse the tools or take shortcuts.
Educators with good intentions, skills, and powerful tools will always beat ignorance and laziness relying on those same tools.
I wrote about this here: https://news.ycombinator.com/item?id=49685036
Is there anything specific you, as an LLM equivalent of a nuclear physicist (if that’s a fair comparison), can highlight in terms of these imminent dangers?
AI is concerning in some aspects, yes, but the loss of a shared reality has nothing to do with that, and is even more scary.
Or do it as a hobby. I love shooting guns, I certainly wouldn’t shoot people if that was the only way to make a career shooting guns.
Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.
"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.
Like a machinist who made machine cogs for (amongst other machinery) the new fabric looms who noticed that he was making more wheels for more and more looms and wants to stop that job and instead become a cottage clothes maker using the new fabric available in the shops.
I also don’t believe addiction and other psychological exploits are used for the sole purpose of ad revenue. There are other more direct forms of monetization involved, such as microtransactions and subscriptions.
If you really want to blame something, it really all comes down to one word:
Greed.
Advertising has become 'active', very recently in the overall lifespan of advertising. For most of it's lifespan it was passive, it blared out at you from the TV or radio or bus stop. Now, however, it's reaching out and taking pieces of you back for analysis and reconstruction, and on-selling to any other entity interested in doing the same.
The reason is greed, yes.
We can't regulate human greed, but we can regulate the manifestation of its symptoms.
We can also choose not to work towards expanding the capacity of said symptoms.
Not if we need to eat and raise our kids. Then we need careers, which depend on what everyone else is doing, so we can't stop, our own future depends on how competitive we remain in the new context. I see an idealist streak here, some kind of naive platonism that ignores everything runs at a cost and nothing executes for free like platonic heaven. We can't take decisions that contradict our own needs.
Continuing to work for evil people is a choice. Don’t brush it off.
Instagram? Tiktok? Facebook?
Social media feeds off addiction to generate ad revenue.
And how much damage has social media done to political and social institutions and to collective mental health?
Ads on other web properties are not weaponised in the same way.
Eating too much fruits has never been healthy, but gorging oneself on raw sugar is worse. Perfect is the enemy of the good.
On the other hand you have pure speculation.
Not only that, AI has been immensely liberating/empowering for many ordinary people (That's why these AI companies have millions of users).
If you really want to blame something, it really all comes down to one word: substance use disorder.
I suppose yeah... It's just greed and corruption.
The incentive structures are wrong/poor/malignant
But they never did quit saying we are harming children, exploiting people with addiction, getting people addicted to social media, earning money from distributing scam/virus thru ads. These people are highest to the highest hypocrites.
I think people with objections to googles business model simply don’t join the company in the first place. We’re decades from Google+ and other various “oh maybe Google isn’t perfect” events, the advertising is a known entity.
There has been a shift, and people are reacting to it
I understand that some people think AI is completely 100% safe, but you guys also have to be aware that a lot of people disagree with you, otherwise you're going to find their actions very confusing.
Sure, there may be some bad actors who utilize it to hack computer systems.
But most of the negative impact will be the subtle influences on the mind which are hard to identify and quantify.
There's no alignment except the arrow of entropy.
In the old testament God was hard and judging heartless, killing his creation through flood, wanting sacrifices, killed 1st borns, leveled two whole cities Sodom and Ghomora, and was in general an hardhearted God that you should fear.
Though, be prepared: it may not be a sufficiently satisfying answer (but then… you have Isaiah 45:15)
I’m always intrigued by the duality and conflict of “I have free will and must choose to act the right way” and “God has a plan so everything will be alright”.
Either free will allows us to deviate from the plan, or the plan will always be carried out and free will doesn’t really matter in the end. Arguing for God’s plan as the thing that will save us is eerily similar to nihilism, because they’re really arguing none of our actions truly matter.
It’s not uncertainty about the motives; those and the end state are clear. It’s just the details of how we get there (“God’s plan”) that aren’t clear.
It should also be noted that it says in Genesis that G-d promises not to destroy the world ever again like He did with the flood. This is important psychologically because it prevents this type of apocalyptic millenarianist thinking.
Right now AI exists to do two things:
- sidestepping legal and regulatory inconveniences like copyright (which drives a massive chunk of investment the same way crypto did by sidestepping financial regulations, AirBnB did by sidestepping housing regulations, Uber did by sidestepping passenger transport regulations and it and the various delivery and other "gig work" apps did by sidestepping labor laws)
- drastically reducing the cost of (and drastically lowering the bar in terms of skillset and effort for) churning out large amounts of low to medium quality assets/code, saturating various systems designed to pay for intellectual property (e.g. ad or subscription based music/video streaming apps with revenue sharing programs, but also literally any form of digital contract work at almost any scale of business)
Neither of these are helping people (in general) and in fact they're extremely destructive (or "disruptive" if you want to make it sound positive) to the existing systems society relies on because they became feasible and (thanks to massive private subsidies, aka "investments", by VC firms and extremely creative accounting) very affordable extremely quickly.
What's worse - while it seems extremely likely that most of the AI generated "works" aren't actually considered copyrightable works by law and it's entirely possible that the use of copyright-protected works in training data does in fact not violate copyright, this does not in fact create a "world without copyright" because IP laws continue to exist and continue to apply, they just no longer meaningfully protect creative labor (i.e. anyone whose profession relies on producing works, whether that's an artist or a programmer) because anything they produce can be cheaply mass reproduced while they still have to stay clear of claims of anyone with enough funds to enforce rights against them.
Under the current economic system there are exactly four ways to earn money to buy back a share of society's resources to sustain your own life (let alone that of your dependents, e.g. kids or aging/impaired parents/spouses/family members):
1. Simple labor: you generally get paid based on your output or a very low flat day rate conditional to maintaining a measurable level of performance. This is typically "low-skilled" menial production labor. Much of this has been replaced by automation in factories and offices alike and most of what couldn't be automated has been moved into countries with fewer labor protections and a higher supply of desperate people willing to accept what are often bad working conditions because cost of labor is usually one of the biggest contributor to overall production cost. In "the West", this kind of work is now scarce and even when it's relatively safe it's often exhausting and unrewarding. A traditional example would be a seamstress or factory line worker, a more modern example would be someone working in visual effects overseas or content moderation / AI training. How much money you can make is ultimately limited by the physical limitations of how much output you can produce per waking/working hour. For the entrepreneurial-minded this would also include people selling their home-made goods on sites like etsy.
2. Time-based: this can be anything from a masseuse to a lawyer or technician. Unlike simple labor the only relevant metric here is time spent. Although employment contracts may specify a fixed wage or at least a fixed component, the understanding is still that this fixed amount is conditional to a minimum average amount of billable time provided. Most employment contracts specify an expected number of working hours for various reasons but we're specifically talking about jobs where the output from the worker to the customer is "billable time". Arguably m...
- Create powerful AI but don't release it and keep it secret. - Ask this AI to create some sort of virus that can be delivered to every human in the world without them knowing it and this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans. - Release this powerful AI to everyone.
Will that work?
I do believe this is possible for a virus to achieve, I just don't think it's possible without killing the human.
excellent logic, m'lord.
You don't think that there is competitive pressure between Anthropic, OpenAI, Google, the various Chinese model companies, and others, to advance the capabilities of their AIs?
Or you don't think that sufficiently advanced AI can cause (mass) harms?
>Or you don't think that sufficiently advanced AI can cause (mass) harms?
Even a feather can cause mass harm if it's used to sign a declaration of war. Something merely being capable of causing mass harm is not an issue and doesn't mean that the existence of feathers are an issue.
Case in point, that hallucination (which, if it was an evil superintelligence, could have been "strategic") that almost led to US boarding a Chinese ship over suspicions of nuclear weapons: https://www.msn.com/en-gb/news/other/us-military-ai-failure-...
It doesn't have SkyNet, it just has to be WOPR. And it doesn't have to be people who are motivated to cause harm, it just has to be people making consequential decisions who are careless, distracted, paranoid or anxious -- which everybody is at some point or another.
And I almost kill people when stopping at a cross walk. That doesn't mean I harmed a pedestrian. Society is set up to be very robust.
>it just has to be WOPR
Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.
Maybe not you personally, but many, many other people have killed many, many pedestrians, and when they exhibit a pattern of bad driving -- or other deviant behavior -- we take them off the streets. That is an example of society being robust.
Except, over here, we're rushing to make AI, which we know has many deviant behaviors and we know caused harms in many circumstances (starting with AI-assisted suicides), even more powerful AND deploy it in more and more real world systems!
As history and, literally, current events show us again and again, society has failure modes that lead to widespread harm and destruction. We've had world wars and then literally had multiple close calls with nuclear war right after. And then we have all that's going on out there. (In related news, the Pentagon threw a hissy fit because Anthropic would not let them use Claude for autonomous killing machines. Do I have to even mention what they've been up to these days?)
Most of these failures are caused by misaligned incentives and socioeconomic forces. The incentives and forces around AI have hints of many brand new failure modes that we can't even foresee because things are moving so fast.
> Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.
"We're already going down this mountain pass at 200 miles an hour, why slow down now?"
I think we should not just slow down frontier AI development, we should also slow down where and how that AI gets deployed.
There are trillions of dollars at stake here, do you think getting 1000 people on board is that difficult with this level of money? lol
Trope circa 1950 not true, because of trope circa even earlier...
Whatever AI people have been saying for the last 70 years, AI capabilities have improved stupendously in the last 10 years and there is no sign of the rate of improvement slowing down. I would like to deny it, but I can't.
People are using AI.
Your argument is that AI cannot be a serious concern because it has major known limitations. But most of us are using it despite the major known limitations. So the existence of the known limitations is irrelevant to determining what level of harms are possible -- unless the known limitations are directly limiting the harms. In practice, it seems like most current harms, and many of the (not stupid) feared harms, are completely possible despite the known limitations. Sometimes even exacerbated by them. ("Sorry, you told me to never stop your insulin pump, but I stopped your insulin pump while diagnosing a test failure. I will record in my memory file that I must always... hello? user?")
The (not stupid) parenthetical might be too weaselly for you, but that's how I see the "AI is going to rise up and intentionally subjugate or exterminate humanity" brand of doomerism. While I don't think that's strictly impossible, there are so many other ways harms that are much more imminent and much harder to defend against, that it seems foolish (to me!) to worry about those.
If you can stop a problem by pulling out the plug, you don't have a real problem.
If you don't want to pull out the plug, you do have a real problem. (Don't want, or can't afford to, or can't be bothered to so extremely that you never will, or whatever.)
Then they say they are going to keep doing it...
Forgive me for alerting you: they are liars
More importantly, as I tried to make clear in my post, the main reason I quit is not specific fears of AI doom (I'm uncertain about that), but my belief that AI is progressing too fast.
Yet the same 1,000 people are all raising funds to do more on the frontier.
The companies have no tools for this, because there is a chance that the company who solves AI will eat the entire cake. And countries cannot do something about it, because clearly it gives an advantage in warfare. When it comes to weapons everything that is possible will be done, which means everybody needs to do it to not be defense less.
There is nothing we can do anymore. We can only hope that LeCun is right and that LLMs are a dead end and that everyone trying to solve AGI with then is essentially wasting money.
Just a reminder that Hinton left Google from a much higher and more influential perch (not dismissing current author, but, you know), and for similar risk and communication reasons, way back in 2023.
This stuff isn't new - it's just newly breaking through into mainstream conversation.
It is not about objectivity or belief.
It is about being.