Computerphile has a series of videos on AGI that touches on this topic, using the lens of "define a human to a computer," a seemingly simple task with a huge slew of ramifications that come with it. Basically exploring whether or not Asimov's Laws are actually useful or practical (they aren't, and this is a huge reason why).
The short version of the specific video in the series about this explores how edge cases become extremely relevant in this domain, and asks whether or not an AGI would consider these to be people or sufficiently people-like:
* Dead people
* People in vegetative states
* Unborn people
* Simulated human brains
* Sufficiently intelligent animals
And so on. We can't simply list every edge case, that list would be literally endless and the developer is guaranteed to miss one that the AGI will eventually find.
They all come with implications. How can an AGI never harm a human without also accurately predicting extremely discrete details from the future? Any action it takes may cause harm or death to one or two humans 50 years later, and it can't really know for certain. It can probably known if its actions will affect hundreds or thousands, but not one or two.
Suddenly a whole lot of deep moral judgements have to be made, some of them nearly impossible for a human (e.g. the aforementioned unborn human example above). The Three Laws of Robotics unfortunately rely way too much on human intuition, something that is extremely difficult to model in a machine.
I may be remembering incorrectly, but the whole point of that story was how those laws fail spectacularly. That's a critical detail. But I digress.
Clearly an AI designer isn't qualified to just make those judgements on their own, and probably doesn't want to anyway. Really no one, either living, dead, or unborn, is qualified to be the judge and jury in this case, and a committee isn't necessarily better.
It's a fascinating topic, to be sure, but I'll admit that it's not my area of expertise.
Asimov's Three Laws of Robotics are often remembered as being hard set rules, but Asimov more often than not used them as the precursors to murder mysteries. So many of the best of his Three Laws stories are locked room murder mysteries where a robot finds some edge case that no one had yet considered. Especially, the entire concept of the Zeroth law is interesting because it was a loophole defined by robots, and the ethics of which are somewhat debated in the series, but most of that debate is left as an exercise to the reader so readers don't often do it. But even a lot of the very edge cases listed here are part of the meta-ethics debates across the Three Laws stories.
Whether or not the Three Laws are a viable system for AGI governance (and I agree it seems rather unlikely, at least in their simplest define forms most encountered in Asimov's stories, but even the stories would claim that the way they are written in English is not exactly how they are coded in positronic technology, however that may be), the ethics conflicts and edge case debates in the stories are very fascinating reads.
It is somewhat surprising to me how many people seem to think the laws are set in stone and don't realize how many of the stories themselves are secretly ethics debates about edge cases. Just because some of the characters believe the laws to be infallible doesn't mean those characters are right and the laws are infallible. There wouldn't be so many tales about the Three Laws if there weren't so many edge cases and exciting locked room mysteries to explore.
Computers simply do as they are told for better or for worse.
The radar operator at Pearl Harbor misinterpreted the "awfully big flight" of incoming planes as friendly U.S. bombers, not enemy bombers. About 55 minutes later, the attack started.
After this attack people were outraged at the radar technology, describing it as "nothing more than a freak gadget". Many simply wanted the technology gone. Ironically the radar was working fine.
In response, the military created the receiver operating characteristic to identify operators who misinterpret enemy planes as allied planes. Named appropriately, I think.
But, solutions... I don't see how professional ethics could actually be applied productively. And I say this as someone with an engineering background.
If a company's business is building bridges, ethics are an orthogonal concern and hence relatively straightforward to apply. But how could a civil engineer productively apply their ethics to a company who's business it is to build concentration camps? Making sure corners aren't being cut doesn't really address the problem! If it is just one company building camps in a far off land, then sure a review board can apply judgment and sanction their employees from the rest of the industry. But if your domestic society shifts to where building camps is an upstanding economic sector...
Essentially all of the funding for consumer software is based on the assumption of surveillance, period. Low marginal costs means investors desire home runs. Metcalfe's law means the introductory price has to be free. Success means users end up locked in - so even if the consumer-facing front end manages to be profitable on its own, the back end is still able to add to profit through surveillance, or at least store everything until they figure it out.
Even business sectors that should be outside sources of non-surveillance development seem to prefer sitting on the sidelines. For example, it would be pretty straightforward for banks to start offering direct public APIs for customer access to accounts, letting users choose which software they'd like to trust with such data. And then even, as an outside authority, putting restrictions on service providers about data retention. But instead the banks stonewall, and so we get surveillance-service providers like Yodlee filling the gap.
I don't really have solution per se. On a different day I'd go on about end-user Free software. But that's clearly outcompeted for the reasons listed above. When is the last time you saw a GNU TV commercial?
> But if your domestic society shifts to where building camps is an upstanding economic sector...
Is that the case, though? It seems to me that in the last few years the tone has shifted in the opposite direction, where even the mainstream is starting to question the overall utility of social networks and other surveilance-filled software.
Talking sure. But has any of this effectively shifted the investment focus?
Founders have not wanted to be creating surveillance-based companies the whole time. The standard course is for that functionality be added afterwards for monetization. In order for the engineers' ethical judgment to have mattered, they had to have prevented this change a priori by designing the software to treat the future company itself as an attacker. This is very expensive.
Yes exactly. The direction he takes his line of thought in the postscript - expelling individual members of the engineering profession for unethical behavior - is fatuous. The big trends are driven by big money: which business models and technologies get funding.
It's not the engineer whose bug accidentally irradiated a patient that's the problem; it's the VC's investing billions in spying on and shaping human behavior.
Professional Ethics often can't stop companies from being evil (that's for government regulations, sort of, mostly), but they can help individuals stand up against such companies. Most such codes of ethics such as American Society of Civil Engineers Code of Ethics [0] or even the Association for Computing Machinery's Code of Ethics [1] are written to the individual first as a part of your relationship (membership) to the larger society.
Most of the actual effective power of such an organization in an ethics violation is to remove its relationship (membership) with an individual that breaks its code of ethics. In a field like Civil Engineering this has bite because few respectable, non-evil companies are willing to hire Civil Engineers that aren't members in good standing of the ASCE. When was the last time a software job interview asked if you were a member in good standing with the ACM? (Sigh, imagine if that saved you a bunch of time in that job interview, as a civil company wouldn't bother asking an ASCE member a laundry list of basic job knowledge.)
In terms of collective action, the professional society/association can help. If a civil engineer refuses to do a job, or whistleblows on a job in progress, due to it being a potential ethics violation the society (because it is not a union) cannot prevent that particular employer from firing the engineer, but it can: A) make sure that the firing is less of a black mark for other employers by making that information transparent and available to future possible employers, B) notify the remaining members at that company that they may be involved in an ethics violation which could result in loss of their society membership, C) notify potential candidates to that company that the company is involved in ethics violations. Sometimes that collective black list of "no respectable engineer will work for you" can be enough of a deterrent for a company to be less evil.
So Code of Ethics are a good step, if you have an association/society with some bite that would apply if they kicked you out as a member. They can sometimes leverage that collectively for overall good.
Organizations with further more bite are Unions, which can and do set contractual terms with employers that can force the company to adopt the Unions' ethics, and Governments, which can and do set regulations that are the closest we have to enforceable ethics terms on companies because Governments in the worst case have armies with which to enforces them.
Pulling it more specifically to your worries about surveillance, it seems to be almost directly addressed in ACM's Code of Ethics section 2.1. We've established that ACM doesn't have much bite in the tech sector, however, as employers mostly don't care about ACM membership. It would be interesting to see someone try to unionize software developers with a goal of promoting non-surveillance as a first principle, but software developers mostly don't seem to want to unionize because they see themselves as white collar and above such concerns as collective bargaining. (Despite many of Hollywood's unions being quite white collar and proving collective action can be essential, but I digress.) That mostly leaves government regulations. Data and Privacy are certainly things that governments could regulate, given they were shown a clear enough need, but also unlikely in certain political climates and by those that passionately are against regulations, even those designed to enforce ethics and protect privacy.
Professional ethics only work to punish outliers, or at least what the industry wants to publicly consider outsiders. You can't reign in the "defense" industry by creating a code of ethics that discourages making offensive weaponry - it will simply be ignored by the relevant parties!
Posit a new organization that "we all" decide to join and care about, say the Association for Personal Computing Machinery (APCM). It comes into existence, and then declares to Google, Faceboot, Amazon, Cisco, etc that they need to clean up or at least detail who is responsible for their organizations' unethical operations, or all of their employees are going to end up on the APCM shitlist. What happens next?
I described the power hierarchy in my post. If "we all" decided to care about the unethical behavior of Google, Facebook, Amazon, Cisco, et al, it's probably too late to join the ACM or your faux APCM as none of those employers care about industry ethics, won't listen to ethics authorities (at least on the issues in this thread), and aren't focused on hiring ethical employees (or at least, make no current recognition/requirement of existing professional societies such as the ACM).
The next step is that "we all" must decide to join and care about the "Union of Software Workers". Only in collective bargaining might "we all" make a difference in the industry, only in the hard tools of work stoppages and shortages, the hard bargains of strong contracts, the forges of blood, sweat, and tears. History shows unionization often literally involves lots of sacrifices, it's definitely a lot of hard work and a lot of people would need to put their jobs and maybe their lives on the line for the cause.
Of course that step is very hard, so the next most likely step is actually that enough smart people get voted into government that care about this stuff like "we all" do that maybe we can force some regulations on the industry.
If "we all" care enough, we could certainly fight the good fight and try to send smart people to government or the hard work of unionization. In both of those cases, joining the ACM (or a purported APCM) might be a first step in organizing, as they've had multiple decades to start a Code of Ethics for which to build a union or government regulations around, but it's certainly not a sufficient step to solve industry problems at this point, in this current software industry culture that currently sees no value in professional societies, professional ethics, etc.
As long as the software industry is infantilized with the concept of the amateur programmer, the rags to riches "taught myself to code and I need no professional training" story, the "we need to spend an 8 hour day interviewing every candidate on basic technical skills because we don't trust professionals to know the basics and also we want to be 'fair' to random people walking in from the street" attitudes, professional organizations like the ACM cannot have much power in the industry. (Which is it, software industry, are we a white collar job that can respect professional codes of ethics and answer to societies such as the ACM, or are we a blue collar job willing to interview anyone even remotely qualified off of the street, and it is way past time that we unionized? You seem to want it both ways here, Google, Facebook, Amazon, et al.)
> But how could a civil engineer productively apply their ethics to a company who's business it is to build concentration camps?
You don't even need to make the example that extreme. How about architects, structural engineers, civil engineers, etc. all working to design and build for-profit prisons that are intended to incarcerate a large fraction of the population?
Or how about food engineers working to create snacks - of questionable-at-best nutritional value - that are ever more addictive?
Many computer scientists are working in the nascent field of home automation. Surely this technology, with its myriad Internet-connected sensors and voracious logging, has the potential to implement a perfect surveillance state - especially combined with data mining. I would like to think the companies building these systems are going to value user control over all else, but somehow that seems unlikely given current models of funding.
15 comments
[ 2.7 ms ] story [ 46.7 ms ] threadThe short version of the specific video in the series about this explores how edge cases become extremely relevant in this domain, and asks whether or not an AGI would consider these to be people or sufficiently people-like:
* Dead people
* People in vegetative states
* Unborn people
* Simulated human brains
* Sufficiently intelligent animals
And so on. We can't simply list every edge case, that list would be literally endless and the developer is guaranteed to miss one that the AGI will eventually find.
They all come with implications. How can an AGI never harm a human without also accurately predicting extremely discrete details from the future? Any action it takes may cause harm or death to one or two humans 50 years later, and it can't really know for certain. It can probably known if its actions will affect hundreds or thousands, but not one or two.
Suddenly a whole lot of deep moral judgements have to be made, some of them nearly impossible for a human (e.g. the aforementioned unborn human example above). The Three Laws of Robotics unfortunately rely way too much on human intuition, something that is extremely difficult to model in a machine.
I may be remembering incorrectly, but the whole point of that story was how those laws fail spectacularly. That's a critical detail. But I digress.
Clearly an AI designer isn't qualified to just make those judgements on their own, and probably doesn't want to anyway. Really no one, either living, dead, or unborn, is qualified to be the judge and jury in this case, and a committee isn't necessarily better.
It's a fascinating topic, to be sure, but I'll admit that it's not my area of expertise.
Whether or not the Three Laws are a viable system for AGI governance (and I agree it seems rather unlikely, at least in their simplest define forms most encountered in Asimov's stories, but even the stories would claim that the way they are written in English is not exactly how they are coded in positronic technology, however that may be), the ethics conflicts and edge case debates in the stories are very fascinating reads.
It is somewhat surprising to me how many people seem to think the laws are set in stone and don't realize how many of the stories themselves are secretly ethics debates about edge cases. Just because some of the characters believe the laws to be infallible doesn't mean those characters are right and the laws are infallible. There wouldn't be so many tales about the Three Laws if there weren't so many edge cases and exciting locked room mysteries to explore.
The radar operator at Pearl Harbor misinterpreted the "awfully big flight" of incoming planes as friendly U.S. bombers, not enemy bombers. About 55 minutes later, the attack started.
After this attack people were outraged at the radar technology, describing it as "nothing more than a freak gadget". Many simply wanted the technology gone. Ironically the radar was working fine.
In response, the military created the receiver operating characteristic to identify operators who misinterpret enemy planes as allied planes. Named appropriately, I think.
https://en.wikipedia.org/wiki/SCR-270#Use_of_SCR-270_radar_a...
https://en.wikipedia.org/wiki/Receiver_operating_characteris...
But, solutions... I don't see how professional ethics could actually be applied productively. And I say this as someone with an engineering background.
If a company's business is building bridges, ethics are an orthogonal concern and hence relatively straightforward to apply. But how could a civil engineer productively apply their ethics to a company who's business it is to build concentration camps? Making sure corners aren't being cut doesn't really address the problem! If it is just one company building camps in a far off land, then sure a review board can apply judgment and sanction their employees from the rest of the industry. But if your domestic society shifts to where building camps is an upstanding economic sector...
Essentially all of the funding for consumer software is based on the assumption of surveillance, period. Low marginal costs means investors desire home runs. Metcalfe's law means the introductory price has to be free. Success means users end up locked in - so even if the consumer-facing front end manages to be profitable on its own, the back end is still able to add to profit through surveillance, or at least store everything until they figure it out.
Even business sectors that should be outside sources of non-surveillance development seem to prefer sitting on the sidelines. For example, it would be pretty straightforward for banks to start offering direct public APIs for customer access to accounts, letting users choose which software they'd like to trust with such data. And then even, as an outside authority, putting restrictions on service providers about data retention. But instead the banks stonewall, and so we get surveillance-service providers like Yodlee filling the gap.
I don't really have solution per se. On a different day I'd go on about end-user Free software. But that's clearly outcompeted for the reasons listed above. When is the last time you saw a GNU TV commercial?
Is that the case, though? It seems to me that in the last few years the tone has shifted in the opposite direction, where even the mainstream is starting to question the overall utility of social networks and other surveilance-filled software.
Founders have not wanted to be creating surveillance-based companies the whole time. The standard course is for that functionality be added afterwards for monetization. In order for the engineers' ethical judgment to have mattered, they had to have prevented this change a priori by designing the software to treat the future company itself as an attacker. This is very expensive.
It's not the engineer whose bug accidentally irradiated a patient that's the problem; it's the VC's investing billions in spying on and shaping human behavior.
Most of the actual effective power of such an organization in an ethics violation is to remove its relationship (membership) with an individual that breaks its code of ethics. In a field like Civil Engineering this has bite because few respectable, non-evil companies are willing to hire Civil Engineers that aren't members in good standing of the ASCE. When was the last time a software job interview asked if you were a member in good standing with the ACM? (Sigh, imagine if that saved you a bunch of time in that job interview, as a civil company wouldn't bother asking an ASCE member a laundry list of basic job knowledge.)
In terms of collective action, the professional society/association can help. If a civil engineer refuses to do a job, or whistleblows on a job in progress, due to it being a potential ethics violation the society (because it is not a union) cannot prevent that particular employer from firing the engineer, but it can: A) make sure that the firing is less of a black mark for other employers by making that information transparent and available to future possible employers, B) notify the remaining members at that company that they may be involved in an ethics violation which could result in loss of their society membership, C) notify potential candidates to that company that the company is involved in ethics violations. Sometimes that collective black list of "no respectable engineer will work for you" can be enough of a deterrent for a company to be less evil.
So Code of Ethics are a good step, if you have an association/society with some bite that would apply if they kicked you out as a member. They can sometimes leverage that collectively for overall good.
Organizations with further more bite are Unions, which can and do set contractual terms with employers that can force the company to adopt the Unions' ethics, and Governments, which can and do set regulations that are the closest we have to enforceable ethics terms on companies because Governments in the worst case have armies with which to enforces them.
Pulling it more specifically to your worries about surveillance, it seems to be almost directly addressed in ACM's Code of Ethics section 2.1. We've established that ACM doesn't have much bite in the tech sector, however, as employers mostly don't care about ACM membership. It would be interesting to see someone try to unionize software developers with a goal of promoting non-surveillance as a first principle, but software developers mostly don't seem to want to unionize because they see themselves as white collar and above such concerns as collective bargaining. (Despite many of Hollywood's unions being quite white collar and proving collective action can be essential, but I digress.) That mostly leaves government regulations. Data and Privacy are certainly things that governments could regulate, given they were shown a clear enough need, but also unlikely in certain political climates and by those that passionately are against regulations, even those designed to enforce ethics and protect privacy.
[0] http://www.asce.org/code-of-ethics/ [1] https://www.acm.org/about-acm/acm-code-of-ethics-and-profess...
Posit a new organization that "we all" decide to join and care about, say the Association for Personal Computing Machinery (APCM). It comes into existence, and then declares to Google, Faceboot, Amazon, Cisco, etc that they need to clean up or at least detail who is responsible for their organizations' unethical operations, or all of their employees are going to end up on the APCM shitlist. What happens next?
The next step is that "we all" must decide to join and care about the "Union of Software Workers". Only in collective bargaining might "we all" make a difference in the industry, only in the hard tools of work stoppages and shortages, the hard bargains of strong contracts, the forges of blood, sweat, and tears. History shows unionization often literally involves lots of sacrifices, it's definitely a lot of hard work and a lot of people would need to put their jobs and maybe their lives on the line for the cause.
Of course that step is very hard, so the next most likely step is actually that enough smart people get voted into government that care about this stuff like "we all" do that maybe we can force some regulations on the industry.
If "we all" care enough, we could certainly fight the good fight and try to send smart people to government or the hard work of unionization. In both of those cases, joining the ACM (or a purported APCM) might be a first step in organizing, as they've had multiple decades to start a Code of Ethics for which to build a union or government regulations around, but it's certainly not a sufficient step to solve industry problems at this point, in this current software industry culture that currently sees no value in professional societies, professional ethics, etc.
As long as the software industry is infantilized with the concept of the amateur programmer, the rags to riches "taught myself to code and I need no professional training" story, the "we need to spend an 8 hour day interviewing every candidate on basic technical skills because we don't trust professionals to know the basics and also we want to be 'fair' to random people walking in from the street" attitudes, professional organizations like the ACM cannot have much power in the industry. (Which is it, software industry, are we a white collar job that can respect professional codes of ethics and answer to societies such as the ACM, or are we a blue collar job willing to interview anyone even remotely qualified off of the street, and it is way past time that we unionized? You seem to want it both ways here, Google, Facebook, Amazon, et al.)
You don't even need to make the example that extreme. How about architects, structural engineers, civil engineers, etc. all working to design and build for-profit prisons that are intended to incarcerate a large fraction of the population?
Or how about food engineers working to create snacks - of questionable-at-best nutritional value - that are ever more addictive?