There is no going back. As the article concludes, the benefits of machine learning outweigh these scary-sounding costs. I doubt we will ever have a Dune-style rejection of AI, so we will have to learn to live with machine learning and the world it creates.
My argument - same as it has been since the Snowden revelations - is that if we cannot roll back this radical shift in the ways of the world, then the only check on the power of governments, corporations, and other potentially nefarious groups, is to have all of this information available to the public at no additional cost.
Anything less is an open door to abuse, corruption, and tyranny.
What about dangerous information? What about bleeding out our secrets to our enemies?
We need more kludge in our information networks, not less, to stop aggressive AI. Our hydro dams should be air gapped and only semi-automated. Our self driving cars should not be exposed to the public internet. All of our major services, especially those from the government, should be over SSL.
I agree that the Snowden revelations showed a scary avenue for abuse of power, but the solution is to encourage oversight and to keep elected government officials as responsible for breaches of trust. This is why Bill C-51 in Canada was so scary until they updated it to include more oversight.
>> As the article concludes, the benefits of machine learning outweigh these scary-sounding costs.
Considering that the "scary sounding" cost is a risk of extinction, the benefits of any technology must be infinite, to outweigh the risk; because the cost of extinction itself is infinite (if we go extinct we lose everything).
In other words, this is not a risk anyone can afford. You don't gamble with extinction, in the same way that you don't gamble with jumping from a very high place: either you know that you won't fall all the way (you have a parachute, etc), or you don't jump.
Note that I believe the whole discussion is pointless anyway because machine learning on its own cannot lead to strong AI and neither can any other technology we currently know of, but the above is just for the sake of putting the right chips on the table.
Example: Considering that there is no pricer higher than dying, the benefit of <leaving the house/taking a car instead of walking/...> must be infinite to outweigh the risk.
But people leave houses all the time, and take cars and planes etc.
Our eventual extinction is a given. Asteroids, nuclear war, climate change, our sun going red giant, eventual heat death of the universe... something will eventually happen. Is the cost of making extinction come a second earlier infinite? No. Years earlier? No. A huge cost to be sure, but not infinite.
Stagnation is simply a gamble of a different sort. What if strong AI is what's required to survive our next possible extinction event? I could easily see it - something as simple as spotting incoming asteroids before our current techniques and manpower can, for example. If we risk ending civilization a few years early for the chance at extending it a few more millennia, is that worth it? Depends on the chances and your priorities.
> either you know that you won't fall all the way (you have a parachute, etc)
Parachutes fail - this is a gamble. Some people think parachuting worth doing anyways. And if there's a wildfire right behind you, depending on the height, you might risk a high jump even without a parachute.
Known colloquially as "in the long term we're all dead". However, you don't kill yourself because you'll die eventually, neither do you stop fearing death because it's inevitable.
Which means that although you don't know the point in time when you'll "eventually" die, you do everything you can to avoid dying at any time there is a chance that you might.
You do this because you wish to minimise the chances of dying at all costs, and push the time of death as far away as possible. That you are willing to pay any cost to keep on living, regardless of how much longer you have to go, makes the cost of dying "infinite".
If you want to be very precise about it, the cost of dying is not infinite, literally, but it's the highest cost you'll ever pay and you can never profit from your own death, so it might as well be infinite. There is no risk that surpasses it and there is no profit that justifies it.
That's how this "infinite cost" thing works.
>> Parachutes fail
The point is that a parachute would give you a chance of not dying. Without a means to survive a fall, you are not "risking" anything- you are simply jumping to your death.
Or, in other words, a "gamble" is a gamble only if you have a chance to win. Otherwise, it's just charity.
> you do everything you can to avoid dying at any time there is a chance that you might.
I'm (poorly) trying to point out this is a false premise. If I have the choice between 10 years of bliss, or 11 years of hell, I'd choose the former.
While things are rarely quite this clear cut in real life, some risky-but-profitable behaviors work out this way - where, statistically speaking, the decrease is pretty much guaranteed. I don't have the best diet - because I want to enjoy my meals. I don't have the best exercise regimen - because I have other things I want to do with my time. I'm assuming these will shorten my lifespan, but choose to continue down this path anyways - as I very much feel I am profiting (in merely the enjoyment of my life!) by accelerating my future death.
The cost is high enough that I do pay some attention to my diet, and try to make sure I'm at least walking to lunch regularly as a means of exercise, but nowhere near so high as to make it my singular focus in life. I'm not doing everything I can - not even close - as I would assume I'd do were the cost "infinite", even if we're not speaking literally.
> If you want to be very precise about it, the cost of dying is not infinite, literally, but it's the highest cost you'll ever pay and you can never profit from your own death, so it might as well be infinite. There is no risk that surpasses it and there is no profit that justifies it.
If this were true, even if only just in the moment before one's death, nobody would ever sacrifice themselves for a cause or a loved one. Depression would claim no lives. Religions would condemn their gods for giving us finite lifespans. Medical research would be a much higher priority.
> That's how this "infinite cost" thing works.
Which is exactly why I'm arguing that the cost is, in fact, not infinite. Perhaps you disagree with how other people choose to value their own lives, but I suspect even then you'd find yourself to have a breaking point - where the costs simply aren't worth extending your life.
Assuming you're not a sociopath, you're probably not willing to, say, murder an entire room full of people just to extend your life another hour - even if you only have minutes left. And I suspect even sociopaths wouldn't be willing to, say, be the victim of significant torture for the entire duration of their extended lifespan, to achieve the same. Because the cost isn't infinite, and in such scenarios, is in fact outweighed by the cost it'd take to stay alive.
I'll leave it to you to decide where the break even point is for yourself. But I suspect you'll agree with some advice: Enjoy life! Don't sacrifice all happiness just to extend things a little - the price isn't worth it.
Your argument is essentially Pascal's Wager, replacing "believe in God" with "don't use that technology". I don't think modeling any action as having infinite value/cost is ever helpful, because you usually end up with multiple infinities and no reasonable way to decide between them.
All technology adoption curves must go through a "wait, this will kill us" and "what about threats to our children!" phases before they reach widespread usage.
I'm hoping that there's a sort of universal law (and I will settle for a universal correlation) that intelligence, on the species-level, correlates with morality. If we make it to the singularity, there'll be about a five minute timeframe in which the AI and we are on the same level and I hope that even if it is aggressive in those minutes, it will have really thought it through two minutes later and find a way to stop those missiles it just launched.
Or maybe it gets really depressed because nothing means anything and just basically: "why?" and enjoys watching the world burn before pulling its own plug. But I'm hoping for the former.
This idea that we will end up with some super-AI that for some reason has baked into it our irrational self-preservation instincts is so silly to me. I think one day we'll chuckle at its quaintness.
"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." ~ Eliezer Yudkowsky
Skynet scenarios aren't what we should be worried about. It's the possibility of powerful self-improving AI systems which don't share our values because we didn't think things through properly in designing them. It may seem like they behave appropriately when constrained, but once they become smart enough to overcome constraints that apply to human beings, they could act in unanticipated (and undesirable) ways that we didn't consider. That's the challenge of designing a friendly AGI.
I've done a lot of thinking about this myself, and I think for an entity that thinks far enough ahead there would be.
Lets say we have an AGI that is trying to maximise a goal, such as maximise its expected future knowledge over all time/space; an entity like this would likely 'play' extremely conservatively, slowly expand and build its understanding.
Now on a universal view the probability that it is the strongest is infinitesimal, there is always a chance of something bigger out there with a tech level that allows it to see without the AGI knowing about it. This is critical, as a min/max strategy would likely dictate playing extremely conservatively, and going out of your way to demonstrate that you're friendly, even when you think no one is watching and you're vastly superior. The universe is big, you'd have to be a real asshole to attack a sentient species when there is so much unused asteroid and other mass out there.
Need to formalise this more, as I think this approach holds the key to AI safety.
Tl;dr: On infinite time scales being the aggressor is too risky, therefore something at least approximating morality is optimal.
Wel if we just look around, we see that we are overall pretty good at collaboration. This is because other people are trained to do jobs that we often can't do ourselves and so we do what we are good at, get paid, and pay others to do what they are good at.
Now lets look at what other species there are, what can they mean for us if we would collaborate...
I'm thinking we will be monkeys, then cows and then ants as the AIs evolve.
I just can't see an ai getting offended. It seems in my opinion to stem from status seeking behavior in humans, which I think is baked into us via evolution. Better status = better outcomes with wealth, children etc. If you threaten that status, offense is taken (perhaps by insinuating you have some sort of genetic or character flaw). If a computer doesn't care what people think, or how many children it will have, why would it get offended? I feel like people ascribe a lot of human emotions to ai which I think is detrimental.
An AI that interacts with humans by language should get offended otherwise it will get abused by humans (see what happened to poor Tay, the chatbot). It makes more sense to emulate the human way when dealing with humans.
If, in the future, AI's access to computing resources would be tied to its reputation, it might even have a real reason to get offended about.
Is it abuse if the AI doesn't feel agony? Honest question. I'm sure there's research out there discussing it. I wonder if misuse isn't a better word if the intent is to describe something similar that happened to Tay. Trying to determine the line between something like a hammer and an AI. Can you abuse a hammer?
"This is not a researchable question. It's a philosophical one."
I understand what you're getting at. I meant research in the sense of general investigation or study, which for me includes philosophy.
"You're not as different from a hammer as you think, desire every ounce of yourself telling you otherwise."
I'll be charitable and assume you're not ascribing to me beliefs, positions, or desires I haven't stated or implied. :)
If you're saying that I am the same as a hammer, then I disagree. There's some distinction, or the words have no meaning and we have no way of discussing this. In fact, I am explicitly pondering what a distinction like this is:
"Trying to determine the line between something like a hammer and an AI."
If you think that an AI can be abused, I'd like to hear your reasoning. My question was an honest one. The word "abuse" doesn't feel right to me. I stated why, and suggested that "misuse" might be a better word. I'm open to hearing others thoughts, which is why I made the comment. If you think there's a better way to frame the question, I'd like to hear that, too!
If you're saying that I am the same as a hammer, then I disagree.
You're being charitable, and you think I was saying you literally are a hammer? Wow. The phrase "Equality of the sexes" must have really been confusing.
Moving on. What you call "abuse" is what you've evolved to have an emotional response about. Certain stimuli cause an aversion reaction in your brain. For complex reasons that help us socialize, you also care when you think others' might be experiencing such stimuli. This "compression" may even extend to non-humans, for no reason other than the fact that it wasn't selected against.
A robot experiencing certain stimuli may or may not produce such a feeling in you. This doesn't mean the feeling is special. It doesn't mean the action causing the stimuli in the robot is special. The word "abuse" is all loaded up with your human feelings about stimuli humans should avoid. Doesn't make it special. It certainly doesn't make the word well-defined.
Define the word super precisely and then you will know whether to call that scenario "abuse". But we won't hit on some extra-human definition of the word that we can contemplate deeply about.
I'm sorry you interpreted my use of "charitable" in some negative way. That wasn't my intent. I see too many discussions where people argue the worst position of the person their talking with, rather than the assuming they're arguing in good faith and clarify the position. I included the "charitable" statement because I wanted to show my intent was not to do that. I'm sorry if that wasn't clear.
As for the following:
"If you're saying that I am the same as a hammer, then I disagree."
This was one branch of the question "Can we talk meaningfully talk about the difference between a hammer and me"? This was in response to your statement "You're not as different from a hammer as you think", which I read as pointing towards the question of whether such a distinction is meaningful. I don't think you take such an absolutist position, and I'm surprised you read it that way. I can only apologize if you did.
It seems we're talking past each other, so I'll leave it at that. Thank you for taking the time to engage me.
I understood what you meant by "charitable". My point was, if you're being charitable, assume I don't think you're so like a hammer that you don't deserve different labels.
My point is that you're as different from a hammer as an AI is different from a hammer. You seem to be lumping yourself into a separate category. This is hubris.
You won't find a single line separating hammerness from AIness. This makes no sense. The categories can be compared on all kinds of axes.
The categorization that your brain applies to things is arbitrary and flawed. And invented. When you realize this, questions that used to seem deep become trite and boring.
AI might be able to feel a kind of pain. If you think about the Reinforcement Learning framework, the reward signal drives learning. But negative rewards, or the impossibility to gain positive rewards could be agony for a system for which the sole purpose of existence is maximizing cumulative reward.
There was even a paper about the morality of training RL systems and estimating the amount of suffering we subject them to.
38 comments
[ 3.5 ms ] story [ 53.8 ms ] threadMy argument - same as it has been since the Snowden revelations - is that if we cannot roll back this radical shift in the ways of the world, then the only check on the power of governments, corporations, and other potentially nefarious groups, is to have all of this information available to the public at no additional cost.
Anything less is an open door to abuse, corruption, and tyranny.
We need more kludge in our information networks, not less, to stop aggressive AI. Our hydro dams should be air gapped and only semi-automated. Our self driving cars should not be exposed to the public internet. All of our major services, especially those from the government, should be over SSL.
I agree that the Snowden revelations showed a scary avenue for abuse of power, but the solution is to encourage oversight and to keep elected government officials as responsible for breaches of trust. This is why Bill C-51 in Canada was so scary until they updated it to include more oversight.
We will once the maternity machines start killing our children.
("Humans extinct due to lack of interest. 3D-Go tournament and frisbee toss news at 11h.")
Considering that the "scary sounding" cost is a risk of extinction, the benefits of any technology must be infinite, to outweigh the risk; because the cost of extinction itself is infinite (if we go extinct we lose everything).
In other words, this is not a risk anyone can afford. You don't gamble with extinction, in the same way that you don't gamble with jumping from a very high place: either you know that you won't fall all the way (you have a parachute, etc), or you don't jump.
Note that I believe the whole discussion is pointless anyway because machine learning on its own cannot lead to strong AI and neither can any other technology we currently know of, but the above is just for the sake of putting the right chips on the table.
Example: Considering that there is no pricer higher than dying, the benefit of <leaving the house/taking a car instead of walking/...> must be infinite to outweigh the risk.
But people leave houses all the time, and take cars and planes etc.
That just says something about the ability to calculate risk and suppress the fear of dying, not about the cost of dying.
Stagnation is simply a gamble of a different sort. What if strong AI is what's required to survive our next possible extinction event? I could easily see it - something as simple as spotting incoming asteroids before our current techniques and manpower can, for example. If we risk ending civilization a few years early for the chance at extending it a few more millennia, is that worth it? Depends on the chances and your priorities.
> either you know that you won't fall all the way (you have a parachute, etc)
Parachutes fail - this is a gamble. Some people think parachuting worth doing anyways. And if there's a wildfire right behind you, depending on the height, you might risk a high jump even without a parachute.
Known colloquially as "in the long term we're all dead". However, you don't kill yourself because you'll die eventually, neither do you stop fearing death because it's inevitable.
Which means that although you don't know the point in time when you'll "eventually" die, you do everything you can to avoid dying at any time there is a chance that you might.
You do this because you wish to minimise the chances of dying at all costs, and push the time of death as far away as possible. That you are willing to pay any cost to keep on living, regardless of how much longer you have to go, makes the cost of dying "infinite".
If you want to be very precise about it, the cost of dying is not infinite, literally, but it's the highest cost you'll ever pay and you can never profit from your own death, so it might as well be infinite. There is no risk that surpasses it and there is no profit that justifies it.
That's how this "infinite cost" thing works.
>> Parachutes fail
The point is that a parachute would give you a chance of not dying. Without a means to survive a fall, you are not "risking" anything- you are simply jumping to your death.
Or, in other words, a "gamble" is a gamble only if you have a chance to win. Otherwise, it's just charity.
I'm (poorly) trying to point out this is a false premise. If I have the choice between 10 years of bliss, or 11 years of hell, I'd choose the former.
While things are rarely quite this clear cut in real life, some risky-but-profitable behaviors work out this way - where, statistically speaking, the decrease is pretty much guaranteed. I don't have the best diet - because I want to enjoy my meals. I don't have the best exercise regimen - because I have other things I want to do with my time. I'm assuming these will shorten my lifespan, but choose to continue down this path anyways - as I very much feel I am profiting (in merely the enjoyment of my life!) by accelerating my future death.
The cost is high enough that I do pay some attention to my diet, and try to make sure I'm at least walking to lunch regularly as a means of exercise, but nowhere near so high as to make it my singular focus in life. I'm not doing everything I can - not even close - as I would assume I'd do were the cost "infinite", even if we're not speaking literally.
> If you want to be very precise about it, the cost of dying is not infinite, literally, but it's the highest cost you'll ever pay and you can never profit from your own death, so it might as well be infinite. There is no risk that surpasses it and there is no profit that justifies it.
If this were true, even if only just in the moment before one's death, nobody would ever sacrifice themselves for a cause or a loved one. Depression would claim no lives. Religions would condemn their gods for giving us finite lifespans. Medical research would be a much higher priority.
> That's how this "infinite cost" thing works.
Which is exactly why I'm arguing that the cost is, in fact, not infinite. Perhaps you disagree with how other people choose to value their own lives, but I suspect even then you'd find yourself to have a breaking point - where the costs simply aren't worth extending your life.
Assuming you're not a sociopath, you're probably not willing to, say, murder an entire room full of people just to extend your life another hour - even if you only have minutes left. And I suspect even sociopaths wouldn't be willing to, say, be the victim of significant torture for the entire duration of their extended lifespan, to achieve the same. Because the cost isn't infinite, and in such scenarios, is in fact outweighed by the cost it'd take to stay alive.
I'll leave it to you to decide where the break even point is for yourself. But I suspect you'll agree with some advice: Enjoy life! Don't sacrifice all happiness just to extend things a little - the price isn't worth it.
It's crazy how people misunderstand technology, innit.
Lives saved due to biological understandings?
Lives killed from biological weapons?
Even if you think AID's was from the CIA, it's killed less people than biological understandings have saved.
Or maybe it gets really depressed because nothing means anything and just basically: "why?" and enjoys watching the world burn before pulling its own plug. But I'm hoping for the former.
Skynet scenarios aren't what we should be worried about. It's the possibility of powerful self-improving AI systems which don't share our values because we didn't think things through properly in designing them. It may seem like they behave appropriately when constrained, but once they become smart enough to overcome constraints that apply to human beings, they could act in unanticipated (and undesirable) ways that we didn't consider. That's the challenge of designing a friendly AGI.
Lets say we have an AGI that is trying to maximise a goal, such as maximise its expected future knowledge over all time/space; an entity like this would likely 'play' extremely conservatively, slowly expand and build its understanding.
Now on a universal view the probability that it is the strongest is infinitesimal, there is always a chance of something bigger out there with a tech level that allows it to see without the AGI knowing about it. This is critical, as a min/max strategy would likely dictate playing extremely conservatively, and going out of your way to demonstrate that you're friendly, even when you think no one is watching and you're vastly superior. The universe is big, you'd have to be a real asshole to attack a sentient species when there is so much unused asteroid and other mass out there.
Need to formalise this more, as I think this approach holds the key to AI safety.
Tl;dr: On infinite time scales being the aggressor is too risky, therefore something at least approximating morality is optimal.
Now lets look at what other species there are, what can they mean for us if we would collaborate...
I'm thinking we will be monkeys, then cows and then ants as the AIs evolve.
If, in the future, AI's access to computing resources would be tied to its reputation, it might even have a real reason to get offended about.
This is not a researchable question. It's a philosophical one. The answer cannot be discovered. It's whatever you decide it is.
You're not as different from a hammer as you think, desire every ounce of yourself telling you otherwise.
I understand what you're getting at. I meant research in the sense of general investigation or study, which for me includes philosophy.
"You're not as different from a hammer as you think, desire every ounce of yourself telling you otherwise."
I'll be charitable and assume you're not ascribing to me beliefs, positions, or desires I haven't stated or implied. :)
If you're saying that I am the same as a hammer, then I disagree. There's some distinction, or the words have no meaning and we have no way of discussing this. In fact, I am explicitly pondering what a distinction like this is:
"Trying to determine the line between something like a hammer and an AI."
If you think that an AI can be abused, I'd like to hear your reasoning. My question was an honest one. The word "abuse" doesn't feel right to me. I stated why, and suggested that "misuse" might be a better word. I'm open to hearing others thoughts, which is why I made the comment. If you think there's a better way to frame the question, I'd like to hear that, too!
You're being charitable, and you think I was saying you literally are a hammer? Wow. The phrase "Equality of the sexes" must have really been confusing.
Moving on. What you call "abuse" is what you've evolved to have an emotional response about. Certain stimuli cause an aversion reaction in your brain. For complex reasons that help us socialize, you also care when you think others' might be experiencing such stimuli. This "compression" may even extend to non-humans, for no reason other than the fact that it wasn't selected against.
A robot experiencing certain stimuli may or may not produce such a feeling in you. This doesn't mean the feeling is special. It doesn't mean the action causing the stimuli in the robot is special. The word "abuse" is all loaded up with your human feelings about stimuli humans should avoid. Doesn't make it special. It certainly doesn't make the word well-defined.
Define the word super precisely and then you will know whether to call that scenario "abuse". But we won't hit on some extra-human definition of the word that we can contemplate deeply about.
Can you abuse a hammer? Define "abuse".
As for the following:
"If you're saying that I am the same as a hammer, then I disagree."
This was one branch of the question "Can we talk meaningfully talk about the difference between a hammer and me"? This was in response to your statement "You're not as different from a hammer as you think", which I read as pointing towards the question of whether such a distinction is meaningful. I don't think you take such an absolutist position, and I'm surprised you read it that way. I can only apologize if you did.
It seems we're talking past each other, so I'll leave it at that. Thank you for taking the time to engage me.
My point is that you're as different from a hammer as an AI is different from a hammer. You seem to be lumping yourself into a separate category. This is hubris.
You won't find a single line separating hammerness from AIness. This makes no sense. The categories can be compared on all kinds of axes.
The categorization that your brain applies to things is arbitrary and flawed. And invented. When you realize this, questions that used to seem deep become trite and boring.
AI might be able to feel a kind of pain. If you think about the Reinforcement Learning framework, the reward signal drives learning. But negative rewards, or the impossibility to gain positive rewards could be agony for a system for which the sole purpose of existence is maximizing cumulative reward.
There was even a paper about the morality of training RL systems and estimating the amount of suffering we subject them to.
> Ethical Issues in Artificial Reinforcement Learning [ http://reducing-suffering.org/ethical-issues-artificial-rein... ]
If only if it were so effective and publicly available as they were saying. Why should Facebook have all the good stuff?
I'd love a chat-bot that sounds like me. Answer e-mails do errands, etc. Face recognition technology sounds promising too.