What would you suggest engineers who work at Google but not on this do? Your IBM example is actually relatively accurate. IBM also had a lot of people working on things that we're not related to war crimes.
The answer is obvious, but not easy or likely. If Google stood to lose their workforce as a result, this would not be happening. Of course people like comfort and money too much to do that, and they’ll tell themselves it’s not their responsibility and hey, someone else will happily take their job. This is after all, a workforce that has concluded that privacy is something to be monetized and subverted at every turn for similar reasons.
Oh yeah, I totally agree. They just do a very good job at marketing their jobs. I mean, no other company has a movie that revolves entirely around how desirable an internship with them is supposed to be [1].
Hey Dimitry,
Cant wait to see what new project you come up with? Really loved your projects and most importantly detailed write ups. Esp, the one where you booted Linux on a AVR. That was Nuts.
Cheers
Not sure what you consider the difference between a "company" and a "corporation", but I know a lot of people who work for small government contractors that definitely have "souls". Not everybody has the same ethical standpoint on military technology.
I know many people who work for large multi-national defense contractors that definitely have "souls" as well, but hiding behind moral relativism won't save you from the fact that in the real world moral bright lines exist within societal contexts. And that said, I'm fairly confident how history will see our current drone program.
How does history see the Enigma crackers? Thousands of nonconsenting Germans were no doubt killed by the “good guys” as a result of the work done by our computer science heroes.
I'm not sure why you're so dismissive of moral relativism in this context when it's incredibly relevant. Someone who believes that developing and exploring this technology will enable us to save the lives of soldiers and better prepare to encounter the technology in the wild is going to have very different opinions than someone who views it as yet another tool to oppress the third world with. Both people hold ethical beliefs that consider and value humanitarian consequences.
Edit: I was slowed down for posting too fast so I'm adding my reply to IntronExon here:
It's an interesting hypothetical but I'd argue its appeal is mostly in its simplicity with little evidence to support the claim. The instances you've cited happened either due to a failure of accurate intelligence or because intelligence indicated the structure was being used by enemy combatants. It's very possible to arrive at the conclusion that any unnecessary casualties which occurred happened as a result of poor military intelligence which better reconnaissance drones could help with. It's equally possible that someone working on this kind of technology could see its field use against groups like ISIS as evidence that weaponized drones are a useful tool in fighting terrorist organizations. Obviously neither of these will be true in every instance, but without strong evidence it's hard to believe that the matter is as simple as "some people are just unemphatic".
Maybe the real difference is that some of us lack the empathy and imagination to consider the plight of some poor bastardized having their wedding, school, or hospital drone-striked, and some of us do. For those who can’t, this is never an emotional issue, just a sterile cost/benefit analysis. Formthe rest of us, worlds die in those blasts, and were partially responsible, and that matters.
indeed - i'm not clear on the basis for the outrage here.
from what the article here says (i don't know more about it than that), it's not clear to me that google is doing more than selling their cloud video analysis products to a customer for experimental use on unclassified data. and this kind of sale necessarily comes with offers of (paid) support and consulting services because customers need help. one might object to this customer, certainly - but if so, one might ask, to which customers is google's cloud business not supposed to sell services on this basis? google has never pretended that it didn't sell its products to the defense and intelligence agencies at all; keyhole (google earth) was for many years a major product in the GEOINT sector [0].
if one's objection is to folks working on anything relating to a military or intelligence problem, well, a very large fraction of US senior faculty have taken funding from defense agencies to work on unclassified versions of applications: tracking, image/video/text analysis challenges, etc. etc. etc.
Rough guess, it might be that we're less than 10 years removed from "Do no evil" being part of Google's motto. While completely unsurprising, it's still disappointing to see how money wins versus ethics. But hey, shareholder value!
Google officially doesn’t care about opportunities less than $1B in size. I very much doubt this is anywhere near a billion dollar opportunity. My educated guess: they’re trying to sell cloud to government and need a good use case. Cloud deal_could be_ a billion dollar opportunity.
When's the last time Democrats put up much resistance to the war we still haven't declared? My money is on military contracts being reliably lucrative for the foreseeable future regardless of any reasonably possible election outcome in the next few years.
They tried to sell it but nobody wanted it, AFAIK. Note that when they bought Boston Dynamics they made a big deal about not renewing military contracts because they didn't want to be in the business of autonomous weaponry.
Ethically, I wonder if this would be any different if Google built a similar software for consumer applications and licensed it similarly to Android being used for surveillance in both the US and far more paranoid governments.
I would imagine that this allows both Google and the US government to have far more insight and control over the direction of this and possible applications.
Not to dismiss the obvious ethical issues of Google having possibly harmful incentives and having their hands tied by the US government financially and legally, but all things considered, I think the main difference here is that Google is already doing this on a massively large scale for the sake of selling ads. It is possible this can help save lives.
Please don't fall for this rhetoric. The technology helps killing people, not save lives. Saving lives means preventing people from being killed. You can't prove any of the drone strikes ever helped preventing people from being killed. You can prove a lot of people (including people most normal human beings would consider civilians) did get killed. And even if killing the target helped save lives you can't prove those lives couldn't have been saved any other way.
Drone strikes don't save lives. Drone strikes take lives. The reason we use to justify taking those lives is that they might help save other lives. But mostly drone strikes are trading the guaranteed death of foreigners for the possibility of saving American lives.
> You can prove a lot of people (including people most normal human beings would consider civilians) did get killed.
Drone strike have and will continue to take place, no matter what Google does. But as you very correctly mention, as it is right now, many innocent civilians accidentally get killed. Doesn't it then directly follow that if said drone strikes become more accurate, then less people (or innocent civilians) will be killed?
This may get me some downvotes, but I imagine that the same ethical issue exists with any weapon.
Guns and bullets exist to kill, and in both human society and the animal kingdom, elimination of a few people can increase the overall population and livelihood of the entire group.
I think what you're arguing is about effectiveness of those weapons, not their use.
However, I truly believe that the right direction for life on this planet is the elimination of death of people or animals.
Sure, you could take a bullet to every jaywalker or tax cheat, but I think we've learned more productive ways to handle those situations, and I hope/pray that we'll learn new ways in how to handle terrorists and other violent criminals that could lead to their redemption.
But in the meantime, we may have to accept a lesser of two evils while searching for a good, and I think more intelligent and selective killing is better than widespread killing.
My big worry isn't so much the killing part (not because it's foreigners, but because they are hopefully only attacking murderers), but the erasure of civil rights via intelligent surveillance employed in the name of security.
That to me seems far more insidious and dangerous (which of course is easy to say when it's not my life or my family's that's being ended, but I'm speaking collectively rather than individually there).
You're making a good point about handling criminals. The US still has the death penalty. The death penalty exists to "protect the public" but most civilised countries have agreed that it's inhumane compared to other alternatives. Its harm outweighs the good it does. We have also banned other inhumane practices like solitary confinement.
Where you're wrong is that drone strikes are a necessary evil. There is no way to measure the efficacy of drone strikes. Sure, you can determine whether the drone strike destroyed the target and with Google's help you might even get a machine to tell you a confidence level of whether the target was correctly identified, but good luck proving that that drone strike even saved as much as a single life.
What we can measure, however, is the civilian cost. Except of course it's nearly zero because those in charge of the drone strikes also get to define who can be considered a civilian. It's easy to have zero dead civilians when you're the one who gets to decide whether those unidentified adult males caught in the blast radius couldn't also maybe have been terrorists all along.
Thank you for your response and I apologize for the delay in responding.
I think you're using an ineffective metric to argue against something you're against ethically.
Here's what I mean: saying it's benefit is immeasurable is arguing for better metrics. If it could be demonstrated that it did effectively end more terrorists lives and saved civilians, would you be celebrating it?
I think you'd still want us as a society to invest in a more humane way of handling it.
> What we can measure, however, is the civilian cost. Except of course it's nearly zero because those in charge of the drone strikes also get to define who can be considered a civilian.
Which, in my opinion makes it immeasurable, currently. It still sounds like your arguing for more data, not a better approach.
I think obviously drone strikes have a benefit to those who use them, otherwise they wouldnt keep using them out of some philosophical attachment to unmanned aircraft.
I do agree we need more and better metrics and data on both their efficacy and civilian cost, but for me, even better would be a solution that doesn't even need the killing.
But if you're asking for the military to abandon something in favor of a more ethical response, you should use a better argument and show how it will lead to less death overall. Otherwise, they'll dance around the semantics of what's being argued.
"You can't prove any of the drone strikes ever helped preventing people from being killed."
Errr, what? That's like saying that the US stopping the Nazis by waging war and killing some of them didn't for sure save lives. I mean who knows, those Nazis may have just decided to stop killing other people all on their own.
I'm not saying drone strikes are necessarily ethical, but saying that any time you kill someone else, you are for sure killing someone but not for sure saving a life, therefore you can never kill anyone preemptively, goes against pretty much every moral or legal system ever devised.
First of all, the US didn't stop the Nazis. The Allies did. Saying the US stopped the Nazis belittles the sacrifices of all the other great nations that worked together and those that suffered the civilian cost of that wretched war -- and I say that as a German.
Secondly that analogy is incredibly misleading. The Nazis wore uniforms. They were clearly identifiable. Nazi Germany was a formal nation state with a government and institutions. Unless you're going to argue for ethically questionable decisions like the firebombing and carpet bombing we're talking about mostly military engagements against military targets.
But yes, even in a "just war", if you kill someone, you know for a fact you're killing someone and you don't know you're saving lives. It's how you justify the killing but it's not the act itself.
You save a life by pushing someone away out of the way of a falling tree. You don't save a life by killing them. I don't know what is so difficult to understand about this.
This doesn't mean you can never make a moral argument for killing someone or that you can't ever say with high confidence that a killing will likely result in lives being saved, but it's an extremely important distinction.
If a bank robber points a loaded gun at a hostage and threatens to kill them and you have every reason to believe them, it's easily justifiable to kill them rather than wait for them to pull the trigger first.
If a nation state declares war and invades another country, it's easily justifiable to kill their soldiers and lieutenants.
But you're still killing people. Calling it "saving lives" is a moral judgment, not an objective description of what you're doing. It describes the motive, not the action. Using it as an objective description assumes the motive is honest and exclusive.
I'm not sure whether you're aware of this, but nobody ever started or joined a war simply to "save lives", even the US didn't join WW2 until they were directly attacked and decided to retaliate rather than remain isolationist.
Let's be completely speculative and alarmist about the potential of AI: It could be like developing nuclear weapons first; if the US gets military AI wrong, it could quickly become a poor vassal state to the world's new superpower.[0]
People are alarmist and speculative because AI's potential is unknown. If the potential of the new blockchain technology is unknown then you can wait and see what happens. But given the stakes with military AI, you can't take even tiny risks; you can't wait and see if your country will be in history books as an experiment that lasted 250 years.
By declining to help the US military, Google engineers take that risk to a degree. But if they participate then they gain enormous leverage: Given the stakes, the US military can't afford to alienate them. I'd hope they can use their leverage to achieve related goals: Agreements banning the use of this technology against civilians, foreign or domestic, and banning sharing the tech with law enforcement. Leverage Congress into passing privacy and civil rights laws protecting Americans against abuses of the technology.
[0] Note that AI changes things in another way: For all human history, military power was tied to population size. In the future, with the right AI and some underground robot factories, potentially a small country could dominate. Maybe Singapore?
In the modern day, the difference between civilian and terrorist is so blurred (not only in factually but also in time - fon't forget, USA's founding fathers were terrorists, if you asked the UK monarchy) even humans often cannot tell. Do we trust AIs to?
> don't forget, USA's founding fathers were terrorists,
This gets repeated all the time, but that doesn't make it true. When did the founding fathers deliberately do mass killings of civilians? They basically won by fighting pitched battles with the British. They had diplomatic relations with several countries (Denmark and France being the major ones). There was a command structure with clear lines of authority (Continental Congress -> George Washington -> other commanders). In addition, there was at least an attempt made to follow international laws.
They were rebels, yes. However, it was more a coalition of existing colonial governments (especially the elected legislatures) deciding to part ways with the British.
By and large, the terrorists of our times are not at all like that. They are not part of a legitimate civilian government fighting an enemy using a formal military organization.
> This gets repeated all the time, but that doesn't make it true.
You left out the “if you asked the British monarchy” part, which was fairly critical; they were leaders of a rebel movement which clearly included terrorists (both by modern standards and, even more officially, in terms of armed combatants deviating from then-current norms of warfare, by what might be imputed to be the standards of the time, though of course the term “terrorist” had not yet been applied as a proper term for those behind the still-in-the-future Terror in Revolutionary France, much less evolved to it's more general sense.)
There are US-designated "terrorist" groups that don't deliberately target civillians and function in a manner similar to traditional armies. Hezbollah comes to mind.
[0] -> If we're talking robot armies, it will still depend on a "smaller" state's access to resources. You can't build robots out of thin air, unless you plan on hacking yourself one. But that depends on someone building one in the first place.
Not a small country. A large transnational corp instead. Especially a one which has a hand/tentacles deep inside the world wide communication web plus large AI development plus great aspiration of not being evil ... Just imagine yourself in the position of wanting to make the world a better place and actually able to herd the world into that better place.
Yeah there’s a real risk of a blitzkrieg sort of situation where some country builds a drone army that just picks a traditional army apart on the battlefield.
Eventually someone is going to cut the cord and have truly autonomous armed vehicles and whoever does it first is going to have a tremendous tactical advantage.
Why do people only talk about weapons of mass destruction and dominance in these discussions? Why is Big, Bad & Heavy seen as the only way to win wars and worlds?
I'd say the first group who thinks outside that box would have a major advantage:
– Artificial viruses and other biological/nanotechnological infectious agents that only your side has an "antidote" or immunity for, to "cleanly" wipe out everyone else but you, without causing physical damage to cities/infrastructure/terrain.
— Insect-like drones the size of flies or mosquitos that you remotely control to inject poison directly into individual enemy leaders, shutting them down before they can form or command a military opposition.
– Having a permanent presence on the Moon so that some of you can survive in case of the whole world going tits up due to events you cannot control or plan for.
Where you're wrong is that military AI isn't a traditional WMD. WMDs work as a deterrent because they can destroy a large area and kill a lot of people, sure. But the reason the Cold War never went hot is that it's immediately obvious when an atomic bomb is used and (to varying degrees) who used it. Also atomic bombs are notoriously difficult to produce -- sure, states will eventually get there (see NK) but "non-state actors" can't even begin to develop the equipment to produce them.
Military AI... not so much. Sure, you might have an idea who hates you at any given moment and who has the means, but it's far fuzzier than a missile you can neatly track from point A to point B. Plus we already know how easy it is to produce cheap knockoffs.
Unlike nukes, military AI isn't a deterrent. Autonomous weapon systems aren't the new H bombs, they're the new AK47s and Toyota pickup trucks.
Continuing to have worked for Google (as a software engineer, with other options) after 2018 will become like having worked for Uber through 2017 -- an indelible black mark on your resume.
As a US citizen concerned with our bloated military budget but who also wants the US to remain the strongest military nation on Earth, I'd like to cut $100 billion from the $600 billion plus yearly US military budget, and allocate that money to infrastructure and social programs. We would still have by far the largest military budget in the world after this.
Whoever knows about the US military budget, how feasible would this be? What is the bulk of the military budget dedicated to?
The biggest fractions are personnel, operations, and health care (last I checked, the Defense Health Agency spends more than the Marine Corps ... and provides health care for 8.4 million Americans. The ones who are that strongest military). But the DoD budget actually pales in comparison to Medicare and Social Security. The US Government is a well-armed insurance company, not a weapons dealer selling health insurance.
You could cut a number of non-essential items, even if they're not the largest - the US Army is the largest employer of musicians in the country [0], and the entire military spent at least $437M in one year on musicians in 2015 [1].
That still leaves $99,563M. I'm not familiar enough with the details of the defense budget, but when looking at budgets, in general, the focus should be on line items that move the needle significantly.
I guess the point I was trying to make is that there is plenty of room for budget trimming - you could go straight to the budget for fighter jets or soldiers or whatever the big ticket items are, but you're going to get a lot more resistance.
How much of that could actually be cut? Even if you cut all the public relations type performances there's still some things you can't cut (even if you don't send buglers to funerals any more, you still need someone to show up and hit play on a boombox).
Members of the military bands also do regular military stuff- I talked to a member of one of the Marine Corps bands that split his time in Iraq about 50/50 between holding his trumpet and his rifle (escorting supply convoys).
>"Google’s Eric Schmidt summed up the tech industry’s concerns about collaborating with the Pentagon at a talk last fall. “There’s a general concern in the tech community of somehow the military-industrial complex using their stuff to kill people incorrectly,”
I sure am surprised that the discussion has already reached the point where tech companies are debating whether they kill people 'incorrectly'. I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
When people refer to the military industrial complex, they are referring to the businesses like Lockheed Martin, Raytheon, etc. that are the private companies involved in designing and building weapons.
Exactly. Those companies are civilian as google is. Lockheed is next door even! Google is part of the intelligence wing of the MIC. The only controversy is that some of the more Berkeley / Burning Man googlers are surprised to learn that.
I think the abhorrent trend is the outsourcing of the people who should be assassinated, and that those people on the kill list will be selected by an ML model.
Google will just a be a processor in a long chain of steps, the actual killing will get passed to a drone team in Las Vegas.
If Google does a better job of defining who should be killed, and it siphons money into the Bay Area, why shouldn’t we be supportive of google’s involvement?
I agree that perhaps we shouldn’t blow people up overseas, but at least while we are at it, why not make the best decisions to avoid killing the wrong people?
>I agree that perhaps we shouldn’t blow people up overseas, but at least while we are at it, why not make the best decisions to avoid killing the wrong people?
I think you have to begin a little further back in the logic train than that state-sponsored murder for hire with the help of closed-source algorithms is legitimate or acceptable.
Just don't forget to implement the counting of false positives. The current system doesn't work well, given that US still counts al-Mansoura as 30 ISIS fighters killed and not 150 IDPs as per the independent UN commission investigation.
I think that oil was the main cause of wars in world during the last decades. And it is one of the rare subjects where people can act with their wallet: choose EV each time you can, do not use oil for heating.
We're not talking about killing imminent threats for self-defense. We're not talking about bombing military targets in a legal war. We're talking about remote drone strikes against poorly defined "non state actors", typically in civilian areas.
Maybe we shouldn't try to legitimise it and abdicate the moral responsibility to a computer. Maybe we shouldn't mislead drone operators by showing them a popup stat "this blob you're looking at has been identified as a valid target with 90% confidence" so they feel better when pushing the button.
If we were talking about space lasers that make an individual person's head explode and high res camera footage you might have a point. But we're literally talking about lobbing bombs at people in public spaces and doing so based on grainy low res data.
The last thing we need is more talking points to justify this.
World War 1 ended. The entire premise of free society is that war comes to an end, there is a such a thing as peacetime. Somehow that idea has died, and with it morality and social conscience. Comparing our spat with a few mountain dwelling cavemen on cellphones with a global War to threatened the existence of civilization itself is beyond ignorant. It's criminal intellectual negligence.
Please don't post ideological rants to Hacker News. Whatever your point is intended to be, flamebait drowns it out and has the side effect of wrecking this place.
From what I've gathered of the context I think the original presentation is correct. I read it as a joke criticizing an implied absurdity of tech groups being concerned specifically with what we use to kill people rather than whether we should be killing those people to begin with.
>I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
I'm not sure if you're criticizing that this happens at all or if you're criticizing it as a new development. Either way, it's generally been actual killing that's reserved for the state (or its mercenaries). Private organizations building and refining weapons for the state has been a thing in America since the old west.
I think both discussions are worth having. The involvement of private contractors in military affairs seems to have amplified over the last few decades especially as war has become more and more focused on technology.
Then there's I think a unique angle to this specific case. If you're going to work for Lockheed Martin or Blackwater you at least know what you're getting into. Google does not present itself as a military contractor. Did everybody who works at Google really know that their code is used for this? Have they been informed, consulted?
There's something especially weird and shady about the fact that someone writes some tensorflow code for image recognition, goes and gets a smoothy from the office bar while the DoD just hooks itself into the API and bombs the hell out of people at the other end of the world.
That's a lot more opaque and blurs the line between civil and defense work in entirely new ways.
I agree that there's definately a need for more dialogue.
>Then there's I think a unique angle to this specific case. If you're going to work for Lockheed Martin or Blackwater you at least know what you're getting into. Google does not present itself as a military contractor. Did everybody who works at Google really know that their code is used for this? Have they been informed, consulted?
When I was reading the article this part jumped out at me too. It's interesting because it's a debate that ultimately focuses on workers rights vs. the rights of the employer. A callous argument could be made that the political views of the employees shouldn't matter because they're not being paid to be policy experts they're paid to make a product. The counter argument of course would be that, even if the first argument is true, an employee who isn't informed about the nature of their work lacks adequate understanding to decide if they feel strongly enough to quit or if they want to take the job in the first place.
google's cloud couldn't host this customer's production application because (afaik) google can't host classified data.
so then you're talking about the fact that your tech is inherently "dual-use." but that's almost anything in computing, absent a license that prevents it. in the early days of postgres, we were a little weirded out by some of the support requests we got [0], given the university prototype nature of the system.
To me that sounds like a misreading of the statement. It makes more sense to interpret it to the effect that tech companies are worried that "the military-industrial complex is using their stuff to kill people" and that this is incorrect. Google doesn't really specialize in optimizing lethality, but it does make a lot of noise about ethical application of its tech.
> I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
States have contracted out to private individuals and firms for the development of tools for that purpose (and the actual killing) for as long as states as distinct from private individuals exercising power have existed.
Literally all of the equipment used to kill people, from the rifles, to the bullets, to the fighter jets, to the parts to repair the fighter jets, are manufactured by private companies in the United States. So I’m confused by your confusion in thinking that Google’s assistance in the drone program is somehow the first private company to participate in providing goods and services that are directly involved in killing people. Is Google somehow more culpable than the companies that make the bullets or bombs? How is providing software and computing power for that any different? You don’t think a private company writes the software that targets things for jets and missiles?
The majority of which will have been produced by armaments companies.
It would be strange to be outraged if, for example, Lockheed was doing something for the military, as they are primarily a defence company. Some might choose not to work for them for that same reason.
Google are a search and advertising company so it's not terribly surprising there's some negative reactions.
Another thing to consider is how companies like BD get money. A LOT of research money comes through military. It is an unfortunate side of things. Lots of people think the tech is cool and exciting, but stop there. The military says "this is cool and exciting AND we can use it. Here's some money." There will be spinoffs, but this is a big reason a lot of funding comes through the military. They have money.
> I sure am surprised that the discussion has already reached the point where tech companies are debating whether they kill people 'incorrectly'. I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
Sure, it's valid to disagree with Google's involvement in this project, but I find it incredibly hard to justify the claim that private enterprises traditionally don't equip armed forces. Providing technology and hardware to armed forces is something that private companies have been doing for centuries. Heck non-state groups selling arms to armed forces likely predates the existence of states as we know them. Most armies prior to the modern era were militias that privately purchased their own equipment.
This project is "assisting in killing people" to at most the same degree that developing munitions guidance, military radar systems, and sonar are "assisting in killing people" (I'd argue less since this technology is purely about reviewing reconnaissance data, not about the actual deployment of weapons) and private companies have been developing those systems for over a century.
That's not a very apt analogy. Google is perceived as a benevolent or neutral service bordering on a public utility. They collect vast amounts of personal information / intelligence because people willingly give it to them.
Google assisting in drone strikes is like the military using the electrical grid to electrocute people in their homes or delivering lethal poisons via the Starbucks Rewards program.
What if your favorite cereal brand started participating in assassinations? It's not inconceivable to deliver poison that way. What if, due to a software error, instead of terrorist #41253 your daughter was delivered poisoned Cheerios.
There is a huge ethical gap between private arms manufacturing and a non-militarized company with this much reach participating in a non-wartime, "preventative" military campaign.
> What if, due to a software error, instead of terrorist #41253 your daughter was delivered poisoned Cheerios.
Using your analogy, isn't that exactly what the Pentagon is trying to prevent by seeking Google's assistance in more innovative solutions? Otherwise, what's the alternative?
Do I really need to spell out why drone strikes are already ethically questionable? Improving automated recognition will not reduce civilian casualties, it will only help justifying the strikes because instead of having humans say they identified the target based on its height they can say the computer did it (based on "deep learning", i.e. based on its height but with computers).
The analogy is highly misleading. Poisoned Cheerio deliveries would be highly targeted. Drone strikes are not. The only way drone strikes avoid civilian casualties at the moment is by defining casualties as terrorists until proven otherwise (e.g. if they're children) and not having sufficient data to accurately say how many people were killed.
> Google is perceived as a benevolent or neutral service bordering on a public utility.
That is not an accident. They worked overtime to construct that image.
Yet it might come as a little surprise that they are more in bed with the government than Northrop Grumman, Verizon, Goldman and Comcast: https://www.opensecrets.org/lobby/clientsum.php?id=D00006782... they spent about $18M in lobbying last year outspending all those other "evil" companies so to speak.
Schmidt was directly involved in Hillary's campaign and there is a picture circulating of him at one of campaign events wearing a "staff" badge: https://www.politico.com/f/?id=00000158-6a12-d435-a9ff-7f5ae... that coupled with the "killing people the right way" is all you need to know what Alphabet stands for and where it is headed.
> There is a huge ethical gap between private arms manufacturing and a non-militarized company with this much reach participating in a non-wartime
I claim that there isn't in the sense that once a company becomes large enough it will start participating in killing people "the right way" as Schmidt put it. That happens regardless if we still want to have that cutesy, colorful, not-evil search company image. Amazon (AWS) is selling billion dollars worth of services to the CIA, I am sure that's not for cat pictures and buying books. Microsoft's hands I am pretty sure are not exactly clean either, etc.
And let's not forget all those car or clothes manufacturers who shamelessly and without any moral qualms offer their products to whomever who can pay for them, be it a terrorist, a government agent, or a common run-of-the-mill criminal, each of who then goes to perform their horrible acts, aided by the purchased product.
These manufacturers are playing both sides and should definitely be kept an eye on, I tell you!
> And let's not forget all those car or clothes manufacturers who shamelessly and without any moral qualms
Good point. Not just moral legal as well. There is a whole bunches countries companies cannot do business with because of embargoes. In many cases it is because of terrorism. Depending on what technology there are also export controls to watch out for.
From Post-WW2 to 90's Gulf War Harley-Davidson built bomb parts in the same factories the bikes were built.
Why are you not outraged about this and saying things like 'Harley assisted the military in killing humans for the past 70 years with metal casings that may be used on your softail?'
I mean why stop at google? The private industry has always been supplying the government with goods and services, some of which go to the military.
Because the precision scored Tomahawk shell casings they produced caused a net decrease in lives lost, or were used to kill members of an utterly vile regime's military.
> This project is "assisting in killing people" to at most the same degree that developing munitions guidance, military radar systems, and sonar are "assisting in killing people"
Which incidentally, many developers find completely abhorrent.
Paradoxically, Google getting involved has the potential, if not the actual goal, of decreasing the amount of mistaken or misidentified targets.
I understand some "conscientious" Googlers feeling uneasy about Google getting involved in this sort of thing, but paradoxically, Google (or other AI resources) getting involved will very likely result in fewer incidents of hitting the wrong targets. In other words a measurable reduction in "collateral casualties/damage"
Our drones are already exceedingly good at hitting the targets they intend to. The problem is that most of the targets they are trying to hit are innocent.
Wouldn't better analysis provided by [AI-]assisted intelligence result in better identification of what things are the correct would-be-targets?
I don't mean getting better at hitting coordinate x,y,z on the mark better but rather identifying which coordinates are actually "real" war targets [munitions factory, command centers, training centers, etc.]
I mean, we literally target community leaders whose sole interaction with terrorists is making sure their village/families aren't in the crossfire. It's not that we don't know this, it's just that we're America, and we don't actually care.
Suffice to say, if Google was deciding who to shoot at, and it came up with a smaller number than they do now, the US would consider that bad design.
I would believe that one of our military's lesser goals (obvs winning and preventing our deaths are the major goals) is to avoid collateral damage.
We are not Hezbollah or Russia or Assad. None of them care much about collateral damage. Pretty sure a few of the above believe collateral damage sows fear and provides them advantage, if short-term.
Here's the thing, even if we magically only killed the people we want to kill, most of whom probably are innocent, and none of which were given a fair trial: We actually still create more violence.
Right now, to a kid growing up in many of these countries, the face of America is our President (...) and the sound of an American drone on the horizon. Now, one of these days, one of those drones pops his dad. In twenty years, that kid is a terrorist.
Our drone program is basically, the exact thing you'd do if you wanted to guarantee that there'd still be terrorists in the Middle East in two decades, because we're practically founding those terrorist groups right now.
What's the alternative? Let them play out their wars in their own style (aka more Assad-style conflicts but with Iran and Saudi as well as Turkey and Israel doing the business of war)?
While I think GWB and Obama both did us a great disservice in stirring up things in the ME. We also cannot just let them "duke it out". It'd be 10x-50x worse.
Pulling out now --would exacerbate the issue incredibly. Only people on the extreme right and extreme left believe that to be our best option.
We killed many, many Germans and many Japanese, in indiscriminate carpet bombing campaigns. Not many of those kids with "dead dads" became terrorists.
The USSR did some pretty nasty things in Afghanistan. You don't have Afghani terrorists going over to Russia and getting theirs.
You'd believe based on what? US doesn't do any on-the-ground post airstrike investigations most of the time, talking to the affected people. So it can't even know what is collateral damage and what is not.
Really, I'd like to see how would a feedback loop that would make US military minimize civcas work in the real world with the current way these things are evaluated by the military.
US had for a long time literaly two people dedicated to processing the hunderds of their civilian casualty incidents reports until recently. They have backlog of a year or so of reports. So after killing thousands of people in Raqqa and leveling the City, US military hasn't even started processing complaints. The operation is over for 5 months and official casualty numbers are like "perhaps we might have killed 2 civilians". "Sorry we have backlog." Meanwhile, hunderd corpses are still being extracted monthly from the leveled buildings in the city months later.
Still, it's mildly better than the idiotic european allies who still maintain that they killed no civilian after droping thousands of ~200kg bombs since 2014. It's really that demented.
It may be a stated goal to minimize collateral damage. But with this crap going on, I don't believe it has any meaning. US military probably employs 100x more people to clean the fucking toilets than to process the civcas reports. Lip service and all.
Wait, what makes you think the US is using drone strikes against "real" war targets? The targets of most drone strikes aren't a formal military, they're "militants".
The problem isn't finding locations to bomb. The problem is being able to tell whether the blob on the screen is the target you want to kill. If any civilians get killed in the process, they're probably terrorists too. If they're identifiably not, well too bad they decided to hang around in a "war zone".
The problem isn't identifying targets. The problem is not killing civilians. Increasing the confidence of identifying targets may actually make things worse by allowing to justify drone strikes that would otherwise have to be called off.
The military likely won't be interested in identifying civilians. They have a list of people to kill and want to know whether one of them is in the target area. If they are and everybody else qualifies as a potential enemy combatant (i.e. they look somewhat male, adult and able-bodied) the drone goes kaboom.
Improving detection won't reduce casualties, it will increase confidence, which will result in more strikes and therefore more casualties. There's no incentive to improve the detection of civilians when everyone who is not obviously a civilian (e.g. children) can be considered a combatant.
This technology won't result in drone strikes being called off because there are civilians in the area. This will result in more drone strikes being made because they can ask a machine to greenlight them instead of having any personal accountability.
> I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
You missed the "using their stuff to" part of that quote, the opportunities for melodrama here are fewer than you might've hoped.
In some way every service you could offer anyone is part of the war machine. When you operate a daycare, you're taking care of somebody's kids, and that somebody either does something for, or is involved with the military. One of the actions typically carried out by an active military is killing people.
Somebody is always using the products of your labour, in some fashion or level of abstraction, to kill people in whatever way their outfit, organization, or mandate deems appropriate.
And if we're to be more honest about what's going on here, you could just as easily say that Google is helping the U.S. government not kill people they never intended to kill. Also, I think it's rather reasonable to assume that the U.S. government's interest in continuing the drone program, or the military operations it replaces, is largely independent of Google's willingness to help them with targeting, which to me indicates that Google has something of a moral obligation to help make that happen if they're called upon to do so.
> I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
Are you from US? You might not be. Sounds like are not aware of the multi-billion dollar (maybe even trillion dollar) military industrial complex.
Also not sure what is meant by "democratic discussion". Who are the constituents here? Everyone, employees, tech people only, HN users?
> kill people 'incorrectly'.
Right. I think that tells us where Google stands. It is already helping kill people, it just helps kill them "correctly" of course. In some kind of a nice, non-evil way presumably
> I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
I agree with you, but I have to point out that nearly all of our wars are motivated to make the world safe for American business. The killing fields run red so that we can mine the green shoots. The last time we voted on military action was right after 9/11, where we gave permission for an unending world-wide war and toppled two countries outright and publicly.
Many defense contractors made a killing off these shenanigans.
I am surprised that this is news. Google has done US government work for a long time. After 9/11 the newly formed homeland security wanted pictures and accurate data in regards to residential addresses in the US. At first they went to the credit bureaus and data brokers to collect and store this information due to the fact they already had researchers on the ground collecting data. Google came along with Google earth and not long after that Google street view. Think about how long it was before they where able to actually monetize google earth & maps. They perfected the art of geodata collection and was likely paid for via government grants [0] and contracts. Even today they sell access to different government agencies[1].
Google Maps was basically pieced together from smaller companies they acquired. From Wikipedia:
Google Earth was originally developed by Keyhole, Inc., a Mountain View-based company founded in 2001. Keyhole, after being spun off from Intrinsic Graphics, received funding from the Central Intelligence Agency's venture capital firm, In-Q-Tel and the National Geospatial-Intelligence Agency, in addition to smaller capital from Nvidia and Sony.
IC-funded startups are often founded by ex-IC people. "Keyhole" was the logical name to choose, since In-Q-Tel is more than familiar with the real deal. it makes the VC pitch much clearer to .gov backers.
Update: Will someone please explain to me why I've been flagged? If this story isn't the most blatant contradiction of their supposed motto, I don't know what is. I feel I'm entirely within my rights as a participant in this community to point that out. Why should I not be allowed to express dissent in reaction to a story like this? Outrageous. Absolutely f--king outrageous.
The issue is that the guidelines ask us to comment civilly and substantively. Drive-by link dropping is not substantive, so if there's something you want to express please write it out.
Alright, then let me spell it out for you: aiding in the development of A.I. enabled drones is evil. This is the slipperiest slope I can imagine and they're already taking the first steps. Where does this all end? We have Facebook and Twitter allowing propagandists to run amok on their platforms. Now, we have Google "analyzing" drone footage for the government. Where does the supposedly virtuous and righteous tech world draw the f--king line?
Google's new cloud offering "flower express" you just type someones personal id, and then the Google database looks up last known position and sends a drone.
So when Damore released his manifesto a lot of people wanted Google to fire him, otherwise they would quit themselves.
Where is the Google employee outrage now? I mean, this right here is bad, it has serious implications. Drone killings are the most outrageous thing the U.S. has done in a while...
It worked for the UK and the British East India company , it should work fine for America. Tesla can set up mining camps on Mars and eliminate ten percent of it's "underperforming" population every year, just like it does at it's factory.
The word used is “outraged” and not “surprised” though. I think that’s more than just a semantic point. In fact there is no mention of surprise in the article. I think that’s a reasonable reaction, whereas surprise is a straw man, largely implying a naïveté which is notably absent in the article.
We faced this at work recently (at our decidely sub-Google scale) when sales guy refused to bid for an opportunity at a weapons manufacturer so we had some interesting discussions around the issue.
Its a little hard to make blanket statements that weapons/warfare are bad. There are good times to use weapons.
An obvious one was at the time of WWII. If the clever people had refused to work on weapons, things would have finished up potentially a lot worse for mankind generally.
And perhaps in our medium term future, as climate change becomes more and more real, a critical mass of people will decry the continued burning of fossil fuels. And if retrograde nations continue to poison our common resource, then maybe some global police force will need weapons to stop them.
There's also an argument to be made that remaining on the cutting edge of military science is a form of national security. Being one of the first, or only, nations to develop and test a new kind of weapon means that you'll also be among the first nations to be able to fully assess its practical viability and evaluate its countermeasures.
I'm not sure I see the relevance of your question since what you're asking for is a necessary quantity not a necessary level of innovation. There could be lots of reasons to develop a better thermonuclear weapon even if it's never put into use. Innovation in that area would help us to gauge the difficulty of miniaturizing such a device and, if it turns out to be difficult but possible, whether or not modern technology makes such a feat possible for a hostile non-nation group such as a terrorist organization. In the situation being discussed in the article, the argument would be that remaining on the cutting edge of drone technology allows us to evaluate the threat that drones could pose to our navy if a situation like the Third Taiwan Strait Crisis ever happened again.
There's also an argument to be made that remaining on the cutting edge of military science is a form of national security.
Your words, I’m asking who’s security? Our security while we pretend to own the world? Our security against invasion? How secure do we need to be, when we can already end all human life in 45 minutes?
If you don't agree with a part of what I've said then you're more than welcome to articulate your objections instead of asking a series of rhetorical questions. It's clear that you have strong feelings on the topic but at this point you're not trying to have a discussion you're just being belligerent.
I’m asking a question you seem unwilling to answer. You invoked national security as a motivation to stay on the cutting edge of military science. I’m asking just what your definition encompasses, and pointing out that it’s clearly not just the sanctity of our borders. Traditionally national security is about the security of a nation, not securing its dominance; the latter goes by a different name. Our vast nuclear arsenal secures the former, but not the latter.
And if you read my replies you'll notice that I gave two examples. In response to your fixation on nuclear weapons I pointed out that developing better designs could be helpful with regard to maintaining an awareness of if modern technology has made some producing some version of the weapon, in my example it was a miniaturized version of the weapon, possible for a terrorist organization. I don't feel like it's asking too much to expect you to realize why that might be a useful thing to know for protecting the nation. In my other example I highlighted that having a cutting edge understanding of drone technology could be useful for defending our navy in the event that we another crisis similar to the time we sent ships to guard Taiwan. Given that the Third Taiwan Strait Crisis happened well within recent history I don't think it's unrealistic to suggest preparing for an analogous situation could be beneficial in the event that it should happen again but escalate this time.
You've posted such a huge number of low-quality comments to Hacker News that I've banned this account.
You also broke the site guidelines repeatedly after we asked you not to, and went into ideological flamewar repeatedly after we asked you not to. I have to conclude that this is not an account that wants to use HN as intended.
You've posted so many low-quality comments and skirted the site guidelines consistently enough that I've banned this account.
Accounts that post prodigious low-quality comments, especially ones that specialize on all the grandiose themes, are particularly damaging to signal/noise ratio, and that is what we care about here.
> Its a little hard to make blanket statements that weapons/warfare are bad.
In the case of the US it's usually the truth. The last 3 major US wars were not fighting Nazis; the last 3 major US wars were mostly imperialistic BS; the last 3 major US wars resulted in 20-30 million killed in 37 nations[1].
Or put another way, just because violent predator X(with a long history of unjustly attacking others) managed to take down another worse violent predator Y doesn't mean you should continue to arm and support violent predator X.
The situation involving the Khmer Rouge is quite complex. They were initially associated with the "Communist" block, but closer to the Chinese side. So after the Soviet/Chinese split, they ended up opposite from the Vietnamese, who where aligned with the Soviet Union. That in turn led to the US supporting them when they went to war with Vietnam. The genocide was pretty much over at that point, since the Khmer Rouge had been in power for a while (those they wanted dead had already been killed) and lost control over most of Cambodia to the Vietnamese.
It was definitely a moral failure on the US side to support a known genocidal group based on "the enemy of my enemy is my friend"-type reasoning, but I don't think you can hold the US directly responsible for the victims that were already dead at the time.
It's fairly obvious that if not for US military activity in the region, US backing of Lon Nol, and the massive covert US bombing campaign in Cambodia, that Cambodia would have never been destabilized to the point that a fanatical regime like the Khmer Rouge could have come to power. A lot of Cambodian people would make that connection too, just so you know.
So let me see if I got this right. The US is in a mighty struggle to stop the expansion of communism, a communist faction takes control of Cambodia and proceeds to kill one in four people, and this is US’s fault?
While I agree that the US has had some bad wars, that article seems a little ... absurd. First line in the tally:
> The U.S. is responsible for between 1 and 1.8 million deaths during the war between the Soviet Union and Afghanistan, by luring the Soviet Union into invading that nation.
Sure, the US did stuff, but you're denying the agency of the Soviet Union (and Afghanistan, for that matter) in actually doing all the killing.
The US has killed enough people directly without blaming it for all armed conflicts since WWII.
> An obvious one was at the time of WWII. If the clever people had refused to work on weapons, things would have finished up potentially a lot worse for mankind generally.
There is still debate about whether the use of nukes was at all justifiable, a lot of the motivation for researchers was to stop the Nazis from getting there first, not beating Japan into submission.
That said, this isn't WW2. The US has the largest military on Earth and the most advanced weapon systems. US drone strikes are killing targets in civilian areas without repercussions. The POTUS has the authorisation to wage war against whomever he chooses as long as he can somehow relate them to someone who was involved in 9/11. Heck, the invasion of Iraq even violated international law with no repercussions.
The US military (CIA included) can pretty much do what it pleases and kill people whenever and wherever it wants. Don't want to kill right now? Off to Guantanamo they go. If they happen to be a US citizen just say they're an enemy combatant and make sure they never see a domestic court of law. Even torture is permissible.
If you're worried the US might not be sufficiently equipped to become a dictatorial global authority, you haven't been paying attention.
Mad respect to sales guy. I'm doing the same and avoiding arms companies (my background is very useful for military research).
I acknowledge some warfare may be legitimate (having been bombed myself), but arms companies don't stop at selling to your personal favorite army which you consider morally right, they keep looking for more business abroad.
I don't want to be the one realizing I'm sitting in a cozy air-conditioned office and having made money from the messed up warfare in some distant far-away country, having a large financial incentive to cause more conflict there.
As long as there's two people left on the planet someone's going to want someone dead. Defense contracts will always be a good investment and it can let you get close to people in power more easily than by lobbying, no wonder Google is taking the opportunity.
175 comments
[ 2.8 ms ] story [ 171 ms ] threadHow very IBM of them.
[1]: http://www.imdb.com/title/tt2234155/
Edit: I was slowed down for posting too fast so I'm adding my reply to IntronExon here:
It's an interesting hypothetical but I'd argue its appeal is mostly in its simplicity with little evidence to support the claim. The instances you've cited happened either due to a failure of accurate intelligence or because intelligence indicated the structure was being used by enemy combatants. It's very possible to arrive at the conclusion that any unnecessary casualties which occurred happened as a result of poor military intelligence which better reconnaissance drones could help with. It's equally possible that someone working on this kind of technology could see its field use against groups like ISIS as evidence that weaponized drones are a useful tool in fighting terrorist organizations. Obviously neither of these will be true in every instance, but without strong evidence it's hard to believe that the matter is as simple as "some people are just unemphatic".
from what the article here says (i don't know more about it than that), it's not clear to me that google is doing more than selling their cloud video analysis products to a customer for experimental use on unclassified data. and this kind of sale necessarily comes with offers of (paid) support and consulting services because customers need help. one might object to this customer, certainly - but if so, one might ask, to which customers is google's cloud business not supposed to sell services on this basis? google has never pretended that it didn't sell its products to the defense and intelligence agencies at all; keyhole (google earth) was for many years a major product in the GEOINT sector [0].
if one's objection is to folks working on anything relating to a military or intelligence problem, well, a very large fraction of US senior faculty have taken funding from defense agencies to work on unclassified versions of applications: tracking, image/video/text analysis challenges, etc. etc. etc.
[0] http://trajectorymagazine.com/genesis-google-earth/
Rough guess, it might be that we're less than 10 years removed from "Do no evil" being part of Google's motto. While completely unsurprising, it's still disappointing to see how money wins versus ethics. But hey, shareholder value!
Look how a mere couple years has changed things.
Maybe I was thinking of Toyota: https://www.pcworld.com/article/3077620/toyota-said-to-be-in...
Not to dismiss the obvious ethical issues of Google having possibly harmful incentives and having their hands tied by the US government financially and legally, but all things considered, I think the main difference here is that Google is already doing this on a massively large scale for the sake of selling ads. It is possible this can help save lives.
Please don't fall for this rhetoric. The technology helps killing people, not save lives. Saving lives means preventing people from being killed. You can't prove any of the drone strikes ever helped preventing people from being killed. You can prove a lot of people (including people most normal human beings would consider civilians) did get killed. And even if killing the target helped save lives you can't prove those lives couldn't have been saved any other way.
Drone strikes don't save lives. Drone strikes take lives. The reason we use to justify taking those lives is that they might help save other lives. But mostly drone strikes are trading the guaranteed death of foreigners for the possibility of saving American lives.
Drone strike have and will continue to take place, no matter what Google does. But as you very correctly mention, as it is right now, many innocent civilians accidentally get killed. Doesn't it then directly follow that if said drone strikes become more accurate, then less people (or innocent civilians) will be killed?
I think what you're arguing is about effectiveness of those weapons, not their use.
However, I truly believe that the right direction for life on this planet is the elimination of death of people or animals. Sure, you could take a bullet to every jaywalker or tax cheat, but I think we've learned more productive ways to handle those situations, and I hope/pray that we'll learn new ways in how to handle terrorists and other violent criminals that could lead to their redemption.
But in the meantime, we may have to accept a lesser of two evils while searching for a good, and I think more intelligent and selective killing is better than widespread killing. My big worry isn't so much the killing part (not because it's foreigners, but because they are hopefully only attacking murderers), but the erasure of civil rights via intelligent surveillance employed in the name of security.
That to me seems far more insidious and dangerous (which of course is easy to say when it's not my life or my family's that's being ended, but I'm speaking collectively rather than individually there).
Where you're wrong is that drone strikes are a necessary evil. There is no way to measure the efficacy of drone strikes. Sure, you can determine whether the drone strike destroyed the target and with Google's help you might even get a machine to tell you a confidence level of whether the target was correctly identified, but good luck proving that that drone strike even saved as much as a single life.
What we can measure, however, is the civilian cost. Except of course it's nearly zero because those in charge of the drone strikes also get to define who can be considered a civilian. It's easy to have zero dead civilians when you're the one who gets to decide whether those unidentified adult males caught in the blast radius couldn't also maybe have been terrorists all along.
I think you're using an ineffective metric to argue against something you're against ethically.
Here's what I mean: saying it's benefit is immeasurable is arguing for better metrics. If it could be demonstrated that it did effectively end more terrorists lives and saved civilians, would you be celebrating it? I think you'd still want us as a society to invest in a more humane way of handling it.
> What we can measure, however, is the civilian cost. Except of course it's nearly zero because those in charge of the drone strikes also get to define who can be considered a civilian.
Which, in my opinion makes it immeasurable, currently. It still sounds like your arguing for more data, not a better approach.
I think obviously drone strikes have a benefit to those who use them, otherwise they wouldnt keep using them out of some philosophical attachment to unmanned aircraft.
I do agree we need more and better metrics and data on both their efficacy and civilian cost, but for me, even better would be a solution that doesn't even need the killing.
But if you're asking for the military to abandon something in favor of a more ethical response, you should use a better argument and show how it will lead to less death overall. Otherwise, they'll dance around the semantics of what's being argued.
Errr, what? That's like saying that the US stopping the Nazis by waging war and killing some of them didn't for sure save lives. I mean who knows, those Nazis may have just decided to stop killing other people all on their own.
I'm not saying drone strikes are necessarily ethical, but saying that any time you kill someone else, you are for sure killing someone but not for sure saving a life, therefore you can never kill anyone preemptively, goes against pretty much every moral or legal system ever devised.
First of all, the US didn't stop the Nazis. The Allies did. Saying the US stopped the Nazis belittles the sacrifices of all the other great nations that worked together and those that suffered the civilian cost of that wretched war -- and I say that as a German.
Secondly that analogy is incredibly misleading. The Nazis wore uniforms. They were clearly identifiable. Nazi Germany was a formal nation state with a government and institutions. Unless you're going to argue for ethically questionable decisions like the firebombing and carpet bombing we're talking about mostly military engagements against military targets.
But yes, even in a "just war", if you kill someone, you know for a fact you're killing someone and you don't know you're saving lives. It's how you justify the killing but it's not the act itself.
You save a life by pushing someone away out of the way of a falling tree. You don't save a life by killing them. I don't know what is so difficult to understand about this.
This doesn't mean you can never make a moral argument for killing someone or that you can't ever say with high confidence that a killing will likely result in lives being saved, but it's an extremely important distinction.
If a bank robber points a loaded gun at a hostage and threatens to kill them and you have every reason to believe them, it's easily justifiable to kill them rather than wait for them to pull the trigger first.
If a nation state declares war and invades another country, it's easily justifiable to kill their soldiers and lieutenants.
But you're still killing people. Calling it "saving lives" is a moral judgment, not an objective description of what you're doing. It describes the motive, not the action. Using it as an objective description assumes the motive is honest and exclusive.
I'm not sure whether you're aware of this, but nobody ever started or joined a war simply to "save lives", even the US didn't join WW2 until they were directly attacked and decided to retaliate rather than remain isolationist.
People are alarmist and speculative because AI's potential is unknown. If the potential of the new blockchain technology is unknown then you can wait and see what happens. But given the stakes with military AI, you can't take even tiny risks; you can't wait and see if your country will be in history books as an experiment that lasted 250 years.
By declining to help the US military, Google engineers take that risk to a degree. But if they participate then they gain enormous leverage: Given the stakes, the US military can't afford to alienate them. I'd hope they can use their leverage to achieve related goals: Agreements banning the use of this technology against civilians, foreign or domestic, and banning sharing the tech with law enforcement. Leverage Congress into passing privacy and civil rights laws protecting Americans against abuses of the technology.
[0] Note that AI changes things in another way: For all human history, military power was tied to population size. In the future, with the right AI and some underground robot factories, potentially a small country could dominate. Maybe Singapore?
This gets repeated all the time, but that doesn't make it true. When did the founding fathers deliberately do mass killings of civilians? They basically won by fighting pitched battles with the British. They had diplomatic relations with several countries (Denmark and France being the major ones). There was a command structure with clear lines of authority (Continental Congress -> George Washington -> other commanders). In addition, there was at least an attempt made to follow international laws.
They were rebels, yes. However, it was more a coalition of existing colonial governments (especially the elected legislatures) deciding to part ways with the British.
By and large, the terrorists of our times are not at all like that. They are not part of a legitimate civilian government fighting an enemy using a formal military organization.
You left out the “if you asked the British monarchy” part, which was fairly critical; they were leaders of a rebel movement which clearly included terrorists (both by modern standards and, even more officially, in terms of armed combatants deviating from then-current norms of warfare, by what might be imputed to be the standards of the time, though of course the term “terrorist” had not yet been applied as a proper term for those behind the still-in-the-future Terror in Revolutionary France, much less evolved to it's more general sense.)
Eventually someone is going to cut the cord and have truly autonomous armed vehicles and whoever does it first is going to have a tremendous tactical advantage.
I'd say the first group who thinks outside that box would have a major advantage:
– Artificial viruses and other biological/nanotechnological infectious agents that only your side has an "antidote" or immunity for, to "cleanly" wipe out everyone else but you, without causing physical damage to cities/infrastructure/terrain.
— Insect-like drones the size of flies or mosquitos that you remotely control to inject poison directly into individual enemy leaders, shutting them down before they can form or command a military opposition.
– Having a permanent presence on the Moon so that some of you can survive in case of the whole world going tits up due to events you cannot control or plan for.
https://www.youtube.com/watch?v=9CO6M2HsoIA
Military AI... not so much. Sure, you might have an idea who hates you at any given moment and who has the means, but it's far fuzzier than a missile you can neatly track from point A to point B. Plus we already know how easy it is to produce cheap knockoffs.
Unlike nukes, military AI isn't a deterrent. Autonomous weapon systems aren't the new H bombs, they're the new AK47s and Toyota pickup trucks.
Whoever knows about the US military budget, how feasible would this be? What is the bulk of the military budget dedicated to?
[0] https://www.npr.org/sections/therecord/2010/09/29/130212353/...
[1] https://www.politico.com/story/2016/05/pentagons-bands-battl...
Members of the military bands also do regular military stuff- I talked to a member of one of the Marine Corps bands that split his time in Iraq about 50/50 between holding his trumpet and his rifle (escorting supply convoys).
>"Google’s Eric Schmidt summed up the tech industry’s concerns about collaborating with the Pentagon at a talk last fall. “There’s a general concern in the tech community of somehow the military-industrial complex using their stuff to kill people incorrectly,”
I sure am surprised that the discussion has already reached the point where tech companies are debating whether they kill people 'incorrectly'. I must have missed the democratic discussion about private businesses assisting in killing people at all, a duty traditionally exercised by states.
Google will just a be a processor in a long chain of steps, the actual killing will get passed to a drone team in Las Vegas.
https://en.wikipedia.org/wiki/Creech_Air_Force_Base
I agree that perhaps we shouldn’t blow people up overseas, but at least while we are at it, why not make the best decisions to avoid killing the wrong people?
I think you have to begin a little further back in the logic train than that state-sponsored murder for hire with the help of closed-source algorithms is legitimate or acceptable.
Maybe we shouldn't try to legitimise it and abdicate the moral responsibility to a computer. Maybe we shouldn't mislead drone operators by showing them a popup stat "this blob you're looking at has been identified as a valid target with 90% confidence" so they feel better when pushing the button.
If we were talking about space lasers that make an individual person's head explode and high res camera footage you might have a point. But we're literally talking about lobbing bombs at people in public spaces and doing so based on grainy low res data.
The last thing we need is more talking points to justify this.
[0]: https://en.wikipedia.org/wiki/IBM_during_World_War_II
We shouldnt be as naive as the Romans.
https://news.ycombinator.com/newsguidelines.html
I'm not sure if you're criticizing that this happens at all or if you're criticizing it as a new development. Either way, it's generally been actual killing that's reserved for the state (or its mercenaries). Private organizations building and refining weapons for the state has been a thing in America since the old west.
Then there's I think a unique angle to this specific case. If you're going to work for Lockheed Martin or Blackwater you at least know what you're getting into. Google does not present itself as a military contractor. Did everybody who works at Google really know that their code is used for this? Have they been informed, consulted?
There's something especially weird and shady about the fact that someone writes some tensorflow code for image recognition, goes and gets a smoothy from the office bar while the DoD just hooks itself into the API and bombs the hell out of people at the other end of the world.
That's a lot more opaque and blurs the line between civil and defense work in entirely new ways.
>Then there's I think a unique angle to this specific case. If you're going to work for Lockheed Martin or Blackwater you at least know what you're getting into. Google does not present itself as a military contractor. Did everybody who works at Google really know that their code is used for this? Have they been informed, consulted?
When I was reading the article this part jumped out at me too. It's interesting because it's a debate that ultimately focuses on workers rights vs. the rights of the employer. A callous argument could be made that the political views of the employees shouldn't matter because they're not being paid to be policy experts they're paid to make a product. The counter argument of course would be that, even if the first argument is true, an employee who isn't informed about the nature of their work lacks adequate understanding to decide if they feel strongly enough to quit or if they want to take the job in the first place.
google's cloud couldn't host this customer's production application because (afaik) google can't host classified data.
so then you're talking about the fact that your tech is inherently "dual-use." but that's almost anything in computing, absent a license that prevents it. in the early days of postgres, we were a little weirded out by some of the support requests we got [0], given the university prototype nature of the system.
[0] https://www.paulaoki.com/.admin/pgapps.html
...what? This is America. Private industry makes all killing machines. Last I checked, Colt, Boeing, Lockheed Martin - all private companies.
"Patches welcome!"
States have contracted out to private individuals and firms for the development of tools for that purpose (and the actual killing) for as long as states as distinct from private individuals exercising power have existed.
It would be strange to be outraged if, for example, Lockheed was doing something for the military, as they are primarily a defence company. Some might choose not to work for them for that same reason.
Google are a search and advertising company so it's not terribly surprising there's some negative reactions.
Sure, it's valid to disagree with Google's involvement in this project, but I find it incredibly hard to justify the claim that private enterprises traditionally don't equip armed forces. Providing technology and hardware to armed forces is something that private companies have been doing for centuries. Heck non-state groups selling arms to armed forces likely predates the existence of states as we know them. Most armies prior to the modern era were militias that privately purchased their own equipment.
This project is "assisting in killing people" to at most the same degree that developing munitions guidance, military radar systems, and sonar are "assisting in killing people" (I'd argue less since this technology is purely about reviewing reconnaissance data, not about the actual deployment of weapons) and private companies have been developing those systems for over a century.
Google assisting in drone strikes is like the military using the electrical grid to electrocute people in their homes or delivering lethal poisons via the Starbucks Rewards program.
What if your favorite cereal brand started participating in assassinations? It's not inconceivable to deliver poison that way. What if, due to a software error, instead of terrorist #41253 your daughter was delivered poisoned Cheerios.
There is a huge ethical gap between private arms manufacturing and a non-militarized company with this much reach participating in a non-wartime, "preventative" military campaign.
Using your analogy, isn't that exactly what the Pentagon is trying to prevent by seeking Google's assistance in more innovative solutions? Otherwise, what's the alternative?
Do I really need to spell out why drone strikes are already ethically questionable? Improving automated recognition will not reduce civilian casualties, it will only help justifying the strikes because instead of having humans say they identified the target based on its height they can say the computer did it (based on "deep learning", i.e. based on its height but with computers).
The analogy is highly misleading. Poisoned Cheerio deliveries would be highly targeted. Drone strikes are not. The only way drone strikes avoid civilian casualties at the moment is by defining casualties as terrorists until proven otherwise (e.g. if they're children) and not having sufficient data to accurately say how many people were killed.
That is not an accident. They worked overtime to construct that image.
Yet it might come as a little surprise that they are more in bed with the government than Northrop Grumman, Verizon, Goldman and Comcast: https://www.opensecrets.org/lobby/clientsum.php?id=D00006782... they spent about $18M in lobbying last year outspending all those other "evil" companies so to speak.
Schmidt was directly involved in Hillary's campaign and there is a picture circulating of him at one of campaign events wearing a "staff" badge: https://www.politico.com/f/?id=00000158-6a12-d435-a9ff-7f5ae... that coupled with the "killing people the right way" is all you need to know what Alphabet stands for and where it is headed.
> There is a huge ethical gap between private arms manufacturing and a non-militarized company with this much reach participating in a non-wartime
I claim that there isn't in the sense that once a company becomes large enough it will start participating in killing people "the right way" as Schmidt put it. That happens regardless if we still want to have that cutesy, colorful, not-evil search company image. Amazon (AWS) is selling billion dollars worth of services to the CIA, I am sure that's not for cat pictures and buying books. Microsoft's hands I am pretty sure are not exactly clean either, etc.
These manufacturers are playing both sides and should definitely be kept an eye on, I tell you!
Good point. Not just moral legal as well. There is a whole bunches countries companies cannot do business with because of embargoes. In many cases it is because of terrorism. Depending on what technology there are also export controls to watch out for.
Why are you not outraged about this and saying things like 'Harley assisted the military in killing humans for the past 70 years with metal casings that may be used on your softail?'
I mean why stop at google? The private industry has always been supplying the government with goods and services, some of which go to the military.
Which incidentally, many developers find completely abhorrent.
I understand some "conscientious" Googlers feeling uneasy about Google getting involved in this sort of thing, but paradoxically, Google (or other AI resources) getting involved will very likely result in fewer incidents of hitting the wrong targets. In other words a measurable reduction in "collateral casualties/damage"
I don't mean getting better at hitting coordinate x,y,z on the mark better but rather identifying which coordinates are actually "real" war targets [munitions factory, command centers, training centers, etc.]
Suffice to say, if Google was deciding who to shoot at, and it came up with a smaller number than they do now, the US would consider that bad design.
We are not Hezbollah or Russia or Assad. None of them care much about collateral damage. Pretty sure a few of the above believe collateral damage sows fear and provides them advantage, if short-term.
Right now, to a kid growing up in many of these countries, the face of America is our President (...) and the sound of an American drone on the horizon. Now, one of these days, one of those drones pops his dad. In twenty years, that kid is a terrorist.
Our drone program is basically, the exact thing you'd do if you wanted to guarantee that there'd still be terrorists in the Middle East in two decades, because we're practically founding those terrorist groups right now.
While I think GWB and Obama both did us a great disservice in stirring up things in the ME. We also cannot just let them "duke it out". It'd be 10x-50x worse.
Pulling out now --would exacerbate the issue incredibly. Only people on the extreme right and extreme left believe that to be our best option.
We killed many, many Germans and many Japanese, in indiscriminate carpet bombing campaigns. Not many of those kids with "dead dads" became terrorists.
The USSR did some pretty nasty things in Afghanistan. You don't have Afghani terrorists going over to Russia and getting theirs.
Really, I'd like to see how would a feedback loop that would make US military minimize civcas work in the real world with the current way these things are evaluated by the military.
US had for a long time literaly two people dedicated to processing the hunderds of their civilian casualty incidents reports until recently. They have backlog of a year or so of reports. So after killing thousands of people in Raqqa and leveling the City, US military hasn't even started processing complaints. The operation is over for 5 months and official casualty numbers are like "perhaps we might have killed 2 civilians". "Sorry we have backlog." Meanwhile, hunderd corpses are still being extracted monthly from the leveled buildings in the city months later.
Still, it's mildly better than the idiotic european allies who still maintain that they killed no civilian after droping thousands of ~200kg bombs since 2014. It's really that demented.
It may be a stated goal to minimize collateral damage. But with this crap going on, I don't believe it has any meaning. US military probably employs 100x more people to clean the fucking toilets than to process the civcas reports. Lip service and all.
The problem isn't finding locations to bomb. The problem is being able to tell whether the blob on the screen is the target you want to kill. If any civilians get killed in the process, they're probably terrorists too. If they're identifiably not, well too bad they decided to hang around in a "war zone".
https://theintercept.com/document/2015/10/14/operation-hayma...
At the very least, they are not yet good at hitting only the intended target.
That's not a standard you should feel proud of.
The military likely won't be interested in identifying civilians. They have a list of people to kill and want to know whether one of them is in the target area. If they are and everybody else qualifies as a potential enemy combatant (i.e. they look somewhat male, adult and able-bodied) the drone goes kaboom.
Improving detection won't reduce casualties, it will increase confidence, which will result in more strikes and therefore more casualties. There's no incentive to improve the detection of civilians when everyone who is not obviously a civilian (e.g. children) can be considered a combatant.
This technology won't result in drone strikes being called off because there are civilians in the area. This will result in more drone strikes being made because they can ask a machine to greenlight them instead of having any personal accountability.
You missed the "using their stuff to" part of that quote, the opportunities for melodrama here are fewer than you might've hoped.
In some way every service you could offer anyone is part of the war machine. When you operate a daycare, you're taking care of somebody's kids, and that somebody either does something for, or is involved with the military. One of the actions typically carried out by an active military is killing people.
Somebody is always using the products of your labour, in some fashion or level of abstraction, to kill people in whatever way their outfit, organization, or mandate deems appropriate.
And if we're to be more honest about what's going on here, you could just as easily say that Google is helping the U.S. government not kill people they never intended to kill. Also, I think it's rather reasonable to assume that the U.S. government's interest in continuing the drone program, or the military operations it replaces, is largely independent of Google's willingness to help them with targeting, which to me indicates that Google has something of a moral obligation to help make that happen if they're called upon to do so.
Are you from US? You might not be. Sounds like are not aware of the multi-billion dollar (maybe even trillion dollar) military industrial complex.
Also not sure what is meant by "democratic discussion". Who are the constituents here? Everyone, employees, tech people only, HN users?
> kill people 'incorrectly'.
Right. I think that tells us where Google stands. It is already helping kill people, it just helps kill them "correctly" of course. In some kind of a nice, non-evil way presumably
I agree with you, but I have to point out that nearly all of our wars are motivated to make the world safe for American business. The killing fields run red so that we can mine the green shoots. The last time we voted on military action was right after 9/11, where we gave permission for an unending world-wide war and toppled two countries outright and publicly.
Many defense contractors made a killing off these shenanigans.
EDIT: Connecting this history to now, recall https://wikileaks.org/google-is-not-what-it-seems/
[0] https://en.wikipedia.org/wiki/Google_Earth#History
[1] https://enterprise.google.com/maps/government/
Google Earth was originally developed by Keyhole, Inc., a Mountain View-based company founded in 2001. Keyhole, after being spun off from Intrinsic Graphics, received funding from the Central Intelligence Agency's venture capital firm, In-Q-Tel and the National Geospatial-Intelligence Agency, in addition to smaller capital from Nvidia and Sony.
https://en.wikipedia.org/wiki/Key_Hole
(But I don't know whether there was any other connection between the company and NRO.)
Update: Will someone please explain to me why I've been flagged? If this story isn't the most blatant contradiction of their supposed motto, I don't know what is. I feel I'm entirely within my rights as a participant in this community to point that out. Why should I not be allowed to express dissent in reaction to a story like this? Outrageous. Absolutely f--king outrageous.
https://news.ycombinator.com/newsguidelines.html
Where is the Google employee outrage now? I mean, this right here is bad, it has serious implications. Drone killings are the most outrageous thing the U.S. has done in a while...
Why are they surprised? Google is the Bell Labs of our time.
https://en.wikipedia.org/wiki/Top_100_Contractors_of_the_U.S...
Its a little hard to make blanket statements that weapons/warfare are bad. There are good times to use weapons.
An obvious one was at the time of WWII. If the clever people had refused to work on weapons, things would have finished up potentially a lot worse for mankind generally.
And perhaps in our medium term future, as climate change becomes more and more real, a critical mass of people will decry the continued burning of fossil fuels. And if retrograde nations continue to poison our common resource, then maybe some global police force will need weapons to stop them.
Your words, I’m asking who’s security? Our security while we pretend to own the world? Our security against invasion? How secure do we need to be, when we can already end all human life in 45 minutes?
https://news.ycombinator.com/newsguidelines.html
You also broke the site guidelines repeatedly after we asked you not to, and went into ideological flamewar repeatedly after we asked you not to. I have to conclude that this is not an account that wants to use HN as intended.
https://news.ycombinator.com/newsguidelines.html
Accounts that post prodigious low-quality comments, especially ones that specialize on all the grandiose themes, are particularly damaging to signal/noise ratio, and that is what we care about here.
https://news.ycombinator.com/newsguidelines.html
In the case of the US it's usually the truth. The last 3 major US wars were not fighting Nazis; the last 3 major US wars were mostly imperialistic BS; the last 3 major US wars resulted in 20-30 million killed in 37 nations[1].
Or put another way, just because violent predator X(with a long history of unjustly attacking others) managed to take down another worse violent predator Y doesn't mean you should continue to arm and support violent predator X.
1. https://www.globalresearch.ca/us-has-killed-more-than-20-mil...
It was definitely a moral failure on the US side to support a known genocidal group based on "the enemy of my enemy is my friend"-type reasoning, but I don't think you can hold the US directly responsible for the victims that were already dead at the time.
> The U.S. is responsible for between 1 and 1.8 million deaths during the war between the Soviet Union and Afghanistan, by luring the Soviet Union into invading that nation.
Sure, the US did stuff, but you're denying the agency of the Soviet Union (and Afghanistan, for that matter) in actually doing all the killing.
The US has killed enough people directly without blaming it for all armed conflicts since WWII.
There is still debate about whether the use of nukes was at all justifiable, a lot of the motivation for researchers was to stop the Nazis from getting there first, not beating Japan into submission.
That said, this isn't WW2. The US has the largest military on Earth and the most advanced weapon systems. US drone strikes are killing targets in civilian areas without repercussions. The POTUS has the authorisation to wage war against whomever he chooses as long as he can somehow relate them to someone who was involved in 9/11. Heck, the invasion of Iraq even violated international law with no repercussions.
The US military (CIA included) can pretty much do what it pleases and kill people whenever and wherever it wants. Don't want to kill right now? Off to Guantanamo they go. If they happen to be a US citizen just say they're an enemy combatant and make sure they never see a domestic court of law. Even torture is permissible.
If you're worried the US might not be sufficiently equipped to become a dictatorial global authority, you haven't been paying attention.
I acknowledge some warfare may be legitimate (having been bombed myself), but arms companies don't stop at selling to your personal favorite army which you consider morally right, they keep looking for more business abroad.
I don't want to be the one realizing I'm sitting in a cozy air-conditioned office and having made money from the messed up warfare in some distant far-away country, having a large financial incentive to cause more conflict there.
This stuff already exists to some extent and can be used by anyone (within Google's terms & services).
https://cloud.google.com/video-intelligence/
“You scratch my back, I scratch yours.”
Same apply to my Google Photos ?