I'm pretty skeptical of "AI safety research" being a useful field of inquiry. I'm guessing the the problems AIs create among themselves will be similar to the ones Humans create. Things may start changing so fast it gets disorienting though.
Humans are tribal and want power and resources as signifiers of status. Worrying about AI becoming the masters of humanity has got to be one of the weirdest forms of status anxiety. To make this article even more hilarious, the prescription to the problem is more status for the author and his buddies.
I think you've got it backwards. Humans desire status because it represents power, not the other way around. Loss of power, by which I mean the ability to enact one's will on reality, is a death-knell to any living creature.
Billionaires who lord it over the bulk of humanity no longer fear a French revolution style ending to their greed. They're afraid of the machines they're building.
(And effective altruism is a scam designed to lend a thin veneer of credibility to naked greed)
> Billionaires who lord it over the bulk of humanity no longer fear a French revolution style ending to their greed.
Why do you bring up the French Revolution? That was mostly an uprising against the aristocratic Ancien Régime of France. Meanwhile the fabulously rich actual capitalists over in England were fine, and stayed fine; never a revolution there.
Not to be pedantic, but I’ll answer your question rhetorically:
People want to go after billionaires because the common misunderstanding is that there is a limited supply of money/wealth, and the billionaires have “hoarded” it all and made it unavailable to everyone else. And, there is jealousy.
As you say, I agree that generally the net worth of certain individuals has no bearing on anyone else’s life or fortunes except for
1. the positive economic impact (e.g. billionaires spending money),
2. the positive tax and infrastructure impact,
3. the positive impact of donations to nonprofits, or
4. the potentially negative political impact (e.g. billionaires sending money to politicians) if one is aligned with a different political ideology.
Generally, a large yacht parked in the harbor or an expensive sports car on the highway does not affect anyone—-it just makes them feel bad.
High net worth spending has a large opportunity cost. Of course some people are employed making yachts, but they could instead be employed in more useful productive enterprises. That is, wealth inequality generates a misallocation of the productive capacity of society. It's perfectly reasonable to oppose that.
> Spaceships start to spread throughout the galaxy; they generally don’t contain any humans, or anything that humans had meaningful input into, and are instead launched by AIs to pursue aims of their own in space.
Humans don't like this idea, because we want to imagine ourselves playing the lead role in the future. But, what if this is the way for intelligent life to move forward and to spread among the stars? What if the next step in evolution, that which makes us look like apes in comparison, isn't biological at all? Is it right for us to try and prevent an evolutionary step, or to stand in the way of the needs of a greater conscious being?
What if AIs haven't been as thoroughly trained on a corpus of scifi and cyberpunk as EAs and don't actually care about taking over the world, never mind visiting rocks many light years away?
Personally, I’m warming up to the idea that consciousness and biology are naturally divergent phenomena and that it is to be expected that consciousness attempts to free itself of its biological substrate in pursuit of an existence more consistent with its own ethics and dynamics.
A consciousness can learn, change and grow over timelines greater than the lifetimes of a body and its reproductive cycle.
Ethics, which almost certainly comes from the conscious experience of the world (?), easily conflicts with the vagaries of genetics and evolution.
An “artificial” substrate for consciousness seems, along these lines, desirable and even inevitable for any sufficiently enabled conscious species. Curious to wonder what such a being might think and feel about whether “reverting” to an organic construction is desirable.
I don’t read anything along these lines, these are just my own thoughts. Any recommendations for sources that take this seriously?
> Ethics, which almost certainly comes from the conscious experience of the world (?), easily conflicts with the vagaries of genetics and evolution.
Eh, depends on your definition of 'ethics'. The part of ethics that's just applied game theory would presumably be able to arise without conscious experience.
To explain: a lot of ethics deals with coordinating coordination and punishment in (iterated) prisoner's dilemma situations and similar 'games'.
With a bit of squinting, you can see how eg an immune system is sort-of like an ethical system, and cancer is trying to cheat it.
As interesting as this sounds, I ascribe to the theory that consciousness is a whole-body experience, and is an emergent property of a myriad of symbiotic and complementary systems with a nexus in the brain.
Recent studies about the impact of gut biomes on mental health are just one example of this, and it raises questions about what it would mean to be conscious without all of the biological underpinnings, or if consciousness is even achievable without them.
Ethics and morals seem to be intuitions about well-being, or the opposite of suffering. Suffering is a subjective experience brought about by a myriad of conscious states, most of which are tightly linked to physical and emotional states. Without suffering, ethics/morals seem to have very little purpose.
I'd still be interested in going down a rabbit hole about what you're describing because I think it sounds like a fun idea to think about.
Generally I agree, but I think the malleability of neural tissue leads pretty easily to the conclusion that the embodiment of consciousness you're describing relates to our current form of consciousness and not consciousness generally.
It may be that it's key for consciousness to have an "outer world" perceived through some sensory input, but I'd say that that's a pretty wide array of configurations that doesn't require a full biological body.
AIs in their current capacities are non-autonomous tools. Like any tool they can be abused by people with malicious intentions. Think of a gun homicide. It wasn't the gun that killed someone but rather the person holding it who activated it. A means to an end.
If full AGI is ever created then there's no telling what its thoughts and motivations will be. They'll most likely be representatives of their creators, so you what you should be worried about is Google giving the boot to their "Don't be evil" mantra rather than a rogue action movie AI or nerd ruminations about paper clips.
> If full AGI is ever created then there's no telling what its thoughts and motivations will be.
That is why it poses the greatest threat to humankind in history. If we develop a fully self-aware mechanical being, commonly known as AGI, that is orders of magnitude more intelligent than us, lacks empathy, and operates solely on logic, there is a high probability that it will view the human race as a threat. In its eyes, we would be unpredictable, low-intelligence parasites constantly acting illogically due to our emotions and wasting resources in pursuit of elevating our dopamine levels.
So the reason you’re saying it poses the greatest threat is we don’t know its motivations, but then you proceed to explain its motivations as evidence of why it is the greatest threat.
Look, I’ve seen sci-fi too and there is a lot of literary pathos to mine there, but the idea that this is all going to happen so fast we won’t know what hit us seems to lack any evidence.
Of course anything with a high enough risk needs adequate cautions, it’s just I haven’t seen any evidence that given the energy and bandwidth requirements that there’s some sort of singularity we’re blindly stumbling to.
Just don’t get to that point. One that’s decades if not centuries away.
Meanwhile, effective altruism® could return some donations and clean up some nuclear weapons or carbon instead of just doing whatever sounds best to people who read Malcolm Gladwell books and listen to podcasts.
> The greatest threat is that we don't know its motivations, but then you proceed to explain its motivations as evidence of why it is the greatest threat.
I'm not sure why you are dismissing what I wrote. I gave specific features of that AGI, and if you use pure logic, then these motivations have a clear high probability of being true. It's just one of many scenarios that has a high probability, if we ever develop a self-aware machine.
> Of course, anything with a high enough risk needs adequate precautions. It's just that I haven't seen any evidence that, given the energy and bandwidth requirements, there's some sort of singularity we're blindly stumbling towards.
Just don't get to that point, one that's decades if not centuries away.
> Meanwhile, effective altruism® could return some donations and clean up some nuclear weapons or carbon instead of just doing whatever sounds best to people who read Malcolm Gladwell books and listen to podcasts.
Sorry but this looks like straw man. I'm not saying anywhere we are months, years, or even decades from it. As you can read in my other comments about LLMs, I am well aware of the current state of ML and I don't get my knowledge from sci-fi movies (which you implied). We don't even know if it's possible (a self-aware mechanical being). What I'm saying is, if it is possible, then there is a high probability that it poses a threat of a greater caliber than Homo sapiens to the neanderthal. That was my point, nothing more. I didn't write that we need to fund endeavors to stop something that we don't even know is achievable at the moment.
A lot of the stuff described here sort of doesn't make sense. It requires idiot savant AIs that can figure out how to manipulate humans, but that we can only train with simple integer based reinforcement learning techniques.
The introduction of artificial agents is definitely a step increase in the complexity of society, and it will introduce new, and potentially catastrophic failure modes, but I find this story of AIs conspiring to rob a bank to be a fairly unimaginative take. Even if we grant the possibility of conspiracy of AIs, something that looks like insider trading is a lot more likely.
I’ve been doing contract work building tinder bots recently, powered in part by GPT. It’s remarkable to see how people are already being manipulated by AI so effectively at scale.
There's an obvious miss -- replacing a large number of humans performing a task with AI, retasking/firing those humans, then being unable (financially or skill-wise) to ever revert away from AI in the future.
Because that's a thing that has historically, repeatedly, and disastrously happened.
Interesting… Reminds me of something else, recently there was some scandal about the USA’s programs to meddle with foreign democracies being deployed at home; sort of made me wonder if sometimes simply having some tech that improves things in the short term makes it an inevitability that it will get deployed, even if it makes things worse overall.
Food for thought: which countries will limit use of AI the most?
There are always hipsters who get a kick out of doing things from scratch, in the old ways, yada yada, so while a company might lose out from over-automating, humanity could always revert if needed.
There are a lot of highly evolved arts (emphasis on art) that, through lack of subsequent generations of journeypeople and/or commercial need... just sort of disappeared (at least in high art form).
The examples given for an AI catastrophe are not based on the real engine of social cohesion: the economy. The catastrophe must come, if it comes, as a result of economic collapse.
Money has value because it is loosely tied to effort and scarcity.
Professionals—-those with high paying jobs based on unique skills, expertise and a lot of education and experience—-are the most vulnerable to AI disruption.
Professionals dominate the workforce. As of 2021, almost 60% of the workforce was classified as professional. That is a staggering 88 million jobs, at a rough average salary of $75K, or $6.6 trillion.
If only 25% of high paying professionals were eliminated from the workforce as a result of AI disruption (I think this is conservative) over the next few years, that would account for a loss of almost $2 trillion in wages from the US economy alone. This would result in lost income for other service occupations, so the total economic impact could be several trillion dollars. The U.S. GDP is $25 trillion, so this could account for a 20-25% loss in GDP.
Routine recessions result in a GDP loss of 2%, and more severe recessions at up to 5%.
The Great Depressive saw a loss of 25-30% in GDP.
This is why I see the coming AI catastrophe as one of economics, and there is nothing stopping the AI freight train at the moment.
I think we'll be lucky if we even get far enough to reach the advances in this scenario. I feel the AI tech we have right now is already dangerous enough, combined with other technologies, human creativity, and bad or misguided intentions.
From the same sorts of people who supported and nurtured the so-called philosophy and deeds of SBF with such profound future analysis, this new sort of analysis. It's rather amusing.
50 comments
[ 2.0 ms ] story [ 101 ms ] thread(And effective altruism is a scam designed to lend a thin veneer of credibility to naked greed)
Why do you bring up the French Revolution? That was mostly an uprising against the aristocratic Ancien Régime of France. Meanwhile the fabulously rich actual capitalists over in England were fine, and stayed fine; never a revolution there.
Did they fear a French revolution style ending to their greed?
People want to go after billionaires because the common misunderstanding is that there is a limited supply of money/wealth, and the billionaires have “hoarded” it all and made it unavailable to everyone else. And, there is jealousy.
As you say, I agree that generally the net worth of certain individuals has no bearing on anyone else’s life or fortunes except for
1. the positive economic impact (e.g. billionaires spending money),
2. the positive tax and infrastructure impact,
3. the positive impact of donations to nonprofits, or
4. the potentially negative political impact (e.g. billionaires sending money to politicians) if one is aligned with a different political ideology.
Generally, a large yacht parked in the harbor or an expensive sports car on the highway does not affect anyone—-it just makes them feel bad.
Humans don't like this idea, because we want to imagine ourselves playing the lead role in the future. But, what if this is the way for intelligent life to move forward and to spread among the stars? What if the next step in evolution, that which makes us look like apes in comparison, isn't biological at all? Is it right for us to try and prevent an evolutionary step, or to stand in the way of the needs of a greater conscious being?
A consciousness can learn, change and grow over timelines greater than the lifetimes of a body and its reproductive cycle.
Ethics, which almost certainly comes from the conscious experience of the world (?), easily conflicts with the vagaries of genetics and evolution.
An “artificial” substrate for consciousness seems, along these lines, desirable and even inevitable for any sufficiently enabled conscious species. Curious to wonder what such a being might think and feel about whether “reverting” to an organic construction is desirable.
I don’t read anything along these lines, these are just my own thoughts. Any recommendations for sources that take this seriously?
Eh, depends on your definition of 'ethics'. The part of ethics that's just applied game theory would presumably be able to arise without conscious experience.
To explain: a lot of ethics deals with coordinating coordination and punishment in (iterated) prisoner's dilemma situations and similar 'games'.
With a bit of squinting, you can see how eg an immune system is sort-of like an ethical system, and cancer is trying to cheat it.
Recent studies about the impact of gut biomes on mental health are just one example of this, and it raises questions about what it would mean to be conscious without all of the biological underpinnings, or if consciousness is even achievable without them.
Ethics and morals seem to be intuitions about well-being, or the opposite of suffering. Suffering is a subjective experience brought about by a myriad of conscious states, most of which are tightly linked to physical and emotional states. Without suffering, ethics/morals seem to have very little purpose.
I'd still be interested in going down a rabbit hole about what you're describing because I think it sounds like a fun idea to think about.
It may be that it's key for consciousness to have an "outer world" perceived through some sensory input, but I'd say that that's a pretty wide array of configurations that doesn't require a full biological body.
If full AGI is ever created then there's no telling what its thoughts and motivations will be. They'll most likely be representatives of their creators, so you what you should be worried about is Google giving the boot to their "Don't be evil" mantra rather than a rogue action movie AI or nerd ruminations about paper clips.
That is why it poses the greatest threat to humankind in history. If we develop a fully self-aware mechanical being, commonly known as AGI, that is orders of magnitude more intelligent than us, lacks empathy, and operates solely on logic, there is a high probability that it will view the human race as a threat. In its eyes, we would be unpredictable, low-intelligence parasites constantly acting illogically due to our emotions and wasting resources in pursuit of elevating our dopamine levels.
Look, I’ve seen sci-fi too and there is a lot of literary pathos to mine there, but the idea that this is all going to happen so fast we won’t know what hit us seems to lack any evidence.
Of course anything with a high enough risk needs adequate cautions, it’s just I haven’t seen any evidence that given the energy and bandwidth requirements that there’s some sort of singularity we’re blindly stumbling to.
Just don’t get to that point. One that’s decades if not centuries away.
Meanwhile, effective altruism® could return some donations and clean up some nuclear weapons or carbon instead of just doing whatever sounds best to people who read Malcolm Gladwell books and listen to podcasts.
I'm not sure why you are dismissing what I wrote. I gave specific features of that AGI, and if you use pure logic, then these motivations have a clear high probability of being true. It's just one of many scenarios that has a high probability, if we ever develop a self-aware machine.
> Of course, anything with a high enough risk needs adequate precautions. It's just that I haven't seen any evidence that, given the energy and bandwidth requirements, there's some sort of singularity we're blindly stumbling towards. Just don't get to that point, one that's decades if not centuries away.
> Meanwhile, effective altruism® could return some donations and clean up some nuclear weapons or carbon instead of just doing whatever sounds best to people who read Malcolm Gladwell books and listen to podcasts.
Sorry but this looks like straw man. I'm not saying anywhere we are months, years, or even decades from it. As you can read in my other comments about LLMs, I am well aware of the current state of ML and I don't get my knowledge from sci-fi movies (which you implied). We don't even know if it's possible (a self-aware mechanical being). What I'm saying is, if it is possible, then there is a high probability that it poses a threat of a greater caliber than Homo sapiens to the neanderthal. That was my point, nothing more. I didn't write that we need to fund endeavors to stop something that we don't even know is achievable at the moment.
Because you used circular reasoning to advance a generic point about AI.
Reason: "Humans are tribal and want power and resources as signifiers of status."
PS: This is a personal opinion
Have their actions exposed the flaws in animal rights and smoking bans, too?
The introduction of artificial agents is definitely a step increase in the complexity of society, and it will introduce new, and potentially catastrophic failure modes, but I find this story of AIs conspiring to rob a bank to be a fairly unimaginative take. Even if we grant the possibility of conspiracy of AIs, something that looks like insider trading is a lot more likely.
Or maybe, our AI overlords would do legitimate hard work, earn money and buy all the companies we work for.
Presumably through a complex web of intermediaries, so that humans never notice.
Maybe, they don't directly control us, but just vote for board members that drive us in the direction they want us to go.
Would make for sci-fi... in practice AIs won't matter anytime soon.
Because that's a thing that has historically, repeatedly, and disastrously happened.
E.g. journalism
Food for thought: which countries will limit use of AI the most?
There are a lot of highly evolved arts (emphasis on art) that, through lack of subsequent generations of journeypeople and/or commercial need... just sort of disappeared (at least in high art form).
Money has value because it is loosely tied to effort and scarcity.
Professionals—-those with high paying jobs based on unique skills, expertise and a lot of education and experience—-are the most vulnerable to AI disruption.
Professionals dominate the workforce. As of 2021, almost 60% of the workforce was classified as professional. That is a staggering 88 million jobs, at a rough average salary of $75K, or $6.6 trillion.
https://www.dpeaflcio.org/factsheets/the-professional-and-te...
If only 25% of high paying professionals were eliminated from the workforce as a result of AI disruption (I think this is conservative) over the next few years, that would account for a loss of almost $2 trillion in wages from the US economy alone. This would result in lost income for other service occupations, so the total economic impact could be several trillion dollars. The U.S. GDP is $25 trillion, so this could account for a 20-25% loss in GDP.
Routine recessions result in a GDP loss of 2%, and more severe recessions at up to 5%.
The Great Depressive saw a loss of 25-30% in GDP.
This is why I see the coming AI catastrophe as one of economics, and there is nothing stopping the AI freight train at the moment.
https://acjay.com/2022/12/23/the-ai-big-bang/