From a normative perspective, terminators are self-evidently dangerous as all hell and should be prohibited.
From a realist frame, I don’t think there will be a deal. Narrow national interests are too divergent w/r/t/ capabilities, intentions, and alternatives. Furthermore, verifibility and consequent trust is weak.
But the reason why higher enforcement doesn't exist is because nobody is trustworthy enough for it. American domination has been relatively good for the West, but citizens of many countries have immensely suffered as a result. Going back further it was European dominance that everyone else suffered under.
Multilateral enforcement becomes yet another tool to take advantage of others that aren't invited.
Are you sure OP's use of "dangerous" wasn't around the implications of fully autonomous, no-human-intervention nature proposed in the article, and not the blatantly literal destructive nature of all war machines?
I think the real reason to hope for some form of prohibition (Geneva Convention) is that Terminators nominally make warfare more palatable than nukes, cruise missiles, or tanks & artillery. Those are obviously untargeted and make for "riveting video" and terrified populations (useful for other purposes of course).
Automated Safety Enforcement Drones are selective and only issue quick justice to the baddies who stand in the way of freedom! If you can't shame leaders based on their prior agreements civil society doesn't have much else to do.
Really, think about what happens when these tools of warfare return home for civilian use, because if they can, they will.
> Terminators nominally make warfare more palatable than nukes, cruise missiles, or tanks & artillery
I guess I’d want to see evidence for this before pursuing an international moratorium everyone will undoubtedly violate the second it’s signed. Remote fire doesn’t seem that much closer than remote kill authorisations.
How can you possibly expect to regulate something like that? Every world power expects that the next major war will be fought with autonomous robots playing a big role. Whoever has the better technology will have a huge advantage. Consequently you can never trust that the other signatories will honor their promises, and so in turn you cannot risk signing it yourself.
Like the Geneva Convention, any robot agreement will be ignored by all combatants as soon as it serves their purposes. War cannot be regulated by gentlemen's agreement.
I much prefer that the state which capriciously claims dominion over the land mass on which I happen to exist today should lose a war with one of its childish rivals than to acquire and advance more deadly toys.
It may not be a gentlemen's agreement, but it's something with which I imagine there are broad consensus across borders.
So basically you don't even value your own life? Is that really the basis of your argument? You don't even try to argue "More deadly weapons are not neccessary for our survival," you just straight up concede "I'm OK with losing the war and having everyone get killed." Good for you, but the rest of us want to survive.
I think it's pretty obvious for those of us living in places "governed" by superpowers that the international affairs of state in which they are currently engaged have no bearing on our individual longevity.
The USA state will fall eventually. Will it be in 500 years? 100? 20? Nobody knows for sure. But the fall will cause the end (or even substantial disruption) of a tiny portion of the lives of the people living in the regime's claimed territory.
I don't fault your logic; I just think it's out-of-date. The world (especially those parts of it we're discussing here) is really quite different than in the 19th century. Very little of the original impetus for the state (specifically, solving problems of coordination created by the absence of established communications across the land-mass, as a non-trivial portion of The Federalist Papers seek to resolve) remains.
In other words; we'll be fine. We won't merely survive.
I'm curious why you assume this. Genocides have largely been limited by human logistics, time, and labor. Given an enormous quantity of morally agnostic and incorruptible autonomous soldiers, why are you so sure that complete genocide would be less likely?
So we avert drone war and consume our way off the environment cliff? Or die before but leave the sludge for the future?
I mean, front row seats to heterogeneous droid swarm mayhem sounds fun.
An “I love my job, disposable consumerism, to prop up other peoples grandparents imagined value to society when many in my family got fucked” is not really helping the “but my American civic life story!” crowd. When it was decided in the 80s business taxes were too high, so sad. So is your so called net worth.
Paraphrasing Thomas Jefferson; the dead do not rule the living.
Madisonian philosophy is that we do.
But Jefferson saw servicing generational story of debt as the whole problem in human history.
Adam Smith also agreed; he wrote the powers that be would need insure equality of condition to avoid manipulation of a free labor market (the only market he wrote of).
Somehow we just get Madisonian policy and tall tails of success in a gender and race biased recent history.
If the US wants respect and drone weapons it’s culture should have to prove it’s as capable of behaving responsibly as any other.
> I think it's pretty obvious for those of us living in places "governed" by superpowers that the international affairs of state in which they are currently engaged have no bearing on our individual longevity.
Wars generally have a deleterious effect to the life expectancy of the civilian population, even more so in the modern era of mechanized warfare where attacking civilian targets at scale is both practical and practiced (usually not by countries that respect the Geneva conventions, but (a) one cannot guarantee that their opponent will be one of those countries and (b) even those countries have been known to set aside parts of those conventions when they perceive either existential threat or expedience).
No, I don't find it likely at all that most people I know live through a hot war on home soil.
You think that security and prosperity is inevitable, but it's really not. World war 2 was less than 100 years ago. There are plenty of people still alive who remember being drafted for Vietnam. There is a 100% chance that there will be another major war at some point. Overall, I fail to see your point.
Have bans on chemical weapons been completely useless, might as well get rid of them all too?
Or nuclear non-proliferation?
Certainly have not been entirely succesful, but I wouldn't want to get rid of them.
As with chemical weapons, they'll still exist, and they'll still be used. But getting the most powerful nations to agree that they are unacceptable and their use has consequences in the international community matters for whether they quickly become pervasive. Vs. allowing the US and Russian defense industries to be in a race to equip the entire world with them as quickly as possible.
This should have ended with the guy on stage getting attacked by a drone deployed by an unknown party, to really drive home the lack of defense against this kind of technology.
I think the key distinction is that, for the most part, the current generation of cruise missiles are targeted and launched with a "human in the loop" - that is, a person actively makes the decision to fire the missile at a specific target. However, the autonomous weapons being debated have significantly less direct human control - they are potentially designed to simply patrol a certain area and attack anything that the weapon classifies as an enemy.
Of course, there are grey areas here - certainly there are existing missiles that can be launched without a defined target and programmed to aim for anything that, say, has radar emissions that match known enemy systems.
Autonomous drones with weapon systems coupled with sufficiently advanced obstacle avoidance, facial recognition, and other AI/ML identification based systems, could swarm cities and eliminate entire battalions or local populations with little to no environmental or infrastructural damage. Imagine Russia preparing to invade the Donbas region of Ukraine and ahead of that invasion deploying a couple of drone swarms to "clear out" a bulk of the Ukranian army presence. Drones could idle, lie in wait and silently hunt opposing forces. It could significantly lower the cost (number of casualties) of invasion for the attacking force.
It is already here. Ukraine got Turkish drones, and Russia can only amass large WWII style force in response which are going to be slaughtered by the drones.
Drones are the new paradigm of warfare which allows smaller players to play in many situations (like the modern limited local wars) on par with the big ones without attempting to acquire nuclear. No UN talks can prevent those smaller players from development of such capabilities.
Yeah I've been kinda wondering about what the deal is there - given the slow public build-up of forces it seems hard to imagine Russia going forward with an old-school invasion when all those troops will just be drone fodder. Is it for internal propaganda? A way to gather intelligence on Ukraine's defensive responses? I'm sure it's hard to tell from the outside.
Ukraine has got strategic advance with the drones (Russia didn't get the message 2 years ago when the air defense Pantsir systems got slaughtered in Syria by those drones). Such a huge advance is though temporary - those Russian air defenses need upgrade to deal with the drones, and it will be done in the near future. So, Ukraine got window of opportunity to solve Donbass the way Azerbajan solved the Karabah using the same drones a year ago. So Russia is trying to prevent it by the public buildup of forces in the way which clearly shows that it can cut Ukranian Donbass forces from the rest of the country by scissor invasion from North and South behind those forces if Ukraine makes a move on Donbass now. Such invasion will be with huge losses and will cost Russia a lot and is very risky, and Russia doesn't want it (at least not at the cost it is going to be at right now), and i'm not sure that Russia can really pull it right now, yet it is the only response Russia got for now, thus such a public build-up of the forces looking a lot like bluffing in poker.
You do the arm signal for drone swarm, shout "drones, drones, drones!", and drop to the ground and cover yourself in your poncho.
My point is that we adapt. It's not worse than chemical weapons and it's preferable to bombing entire cities to achieve your objective. Is it finally dawning on HN that war is hell?
You're missing the point, new technological developments in warfare aren't about "worse or better" since one human wishing to inflict suffering or death on another needs only a sharp stick or a rock and their imagination to produce the worst forms of torture. They're about physical distance. Fighting hand to hand requires an immense level of personal commitment to causing harm. Each technological advance in warfare has lowered that barrier by increasing the distance at which you can maim and kill. It's now possible to kill without ever seeing the person you are killing. Autonomous weapons are the final stage of this development where you don't even need to make the decision to kill, you simply design some inevitably flawed conditions under which you would theoretically kill someone and the machine takes care of the rest.
I think the objective should be preventing war and that means making it more difficult to kill not less.
I agree and that's usually the connotation behind the words, "war is hell." Unfortunately, either the entire world agrees to never weaponize drones or we all need to start thinking about weaponizing drones. That's how arms races work.
If wars are going to happen though I think we are better off with drones rather than annihilating entire cities, civilians and soldiers together. It's important to elect leaders who are not cavalier about deploying drones needlessly.
I suspect ‘killer robots’ wont be banned until after they are used to great effect in some future conflict. Think chemical weapons bans that emerged after WWI.
Chemical weapons were banned because they weren't very effective on the battlefield and generated horrible PR for all involved, so it was an easy win for everyone to ban them.
The problem with autonomous weapon platforms (probably swarms) is that it looks like they're very powerful weapons that give their users advantages over existing capabilities. Enforcing a ban on them will be like enforcing a ban on gunpowder with swords and castles.
If you are a major power with the capability to make 100K+ killer drones or similar per year, going to be hard to get you to sign this.
What is the drones annual salary? What is their annual housing cost, base allowance costs, VA medical benefits (lifetime)? How upset will your population be if one is "killed". Get the beancounters involved, and killer drones are going to be very tempting. For $75B per year or so (10% of US military budget) what kind of killer drones could I buy and how many?
The cost to deploy one F35 is astronomical. The training costs and lifetime costs for the pilots alone is heavy. The F-35 dedicates MASSIVE payload to life support for its human occupant. What kind of killer drones can you build with none of that mass, none of the gforce limits? None of the redundancy requirements? Do you need an aircraft carrier (thousands of sailors! -> needs destroyers and a battle group). No?
Can we design these to drop from space? Park 50K in space for quick de-orbit and insertion globally? Can you do this with people these days easily?
Going to be a scary world.
My quick guess is US, China, Russia, EU (maybe), India as places with enough capacity for this type of next gen battlefield? Maybe EU will move too slowly, so it'll be US / China / Russia?
> If you are a major power with the capability to make 100K+ killer drones or similar per year, going to be hard to get you to sign this.
Surely the individual decision-makers are, as with all human beings, averse to a world swarming with killer robots.
That states make indefensible decisions with much greater frequency than their individual statesfolk is so obvious at this point; it's time to simply stop tolerating their existence and move more swiftly into the next stage of information humanity.
I'm a citizen of one of the states you mention and I can assure you: amidst all the disagreement and political polarization, there is complete consensus on not wanting a world dominated by killer robots.
> there is complete consensus on not wanting a world dominated by killer robots.
That and nukes. It is ok to not want these things, but to survive as a nation state, these things will become necessary developments in order to survive. If GP is correct about some of the capabilities we're likely to see, then any nation state without these will need to begin development in order to ensure survival. Is that an arms race waiting to happen? Yes. Is there a better way? No. The alternative to an escalating arms race is actual war.
We will never be done with power hungry ideologues, we will never be done with Communists whose sole mission is dominion over the whole world, we will never be done with petty despots (looking at you, Putin) who so wish they could play a meaningful role on the world stage. All of these have every reason to develop a new super weapon to tilt the balance of power in their favor.
Iran, for example, has started its centrifuges again. They also have some new ones that spin at 10x the speed of their old ones. They are very close to possessing a nuclear bomb. Up until the end of the Trump presidency they had halted production of uranium and talks were going well. Biden has about the weakest foreign policy of any president in the last half century.
Part of that effective foreign policy is the military might the US has. (Why do we have 5 carrier battle groups? Why does Britain only have one? Why does Russia have a single carrier that's in rather poor repair?)
If killer drones are an effective war-fighting tool (an it appears they are), they are inevitable. What we need to ensure is that they are used as humanely as possible.
Yeah, good point. But I can absolutely envision - without much difficulty - a megatons-to-megawatts motion swiftly bringing an end to this period in Earth's history.
And more generally: maintenance of arsenals of weapons of mass destruction is _very_ costly. Economics of energy production and storage are perhaps a viable mechanism to add pressure in favor of disarmament.
> (Why do we have 5 carrier battle groups? Why does Britain only have one? Why does Russia have a single carrier that's in rather poor repair?)
We can also summarize questions of stagnant geopolitical asymmetry this way:
How long will the internet abide states as nodes of economic and social standing?
Trying to shoehorn a previous world order into an information age society seems like a colossal waste of time in service of nostalgia.
Nostalgia does not guarantee me personal autonomy. Neither does it guarantee me a right to self defense. Nor does it guarantee a free practice of religion, free press, or a right of association.
Ensuring the survival of a nation state that adheres to the US Constitution will guarantee those things.
> How long will the internet abide states as nodes of economic and social standing?
Take a look at China and their authoritarian take on "the internet". "The internet", that is the masses that make it up, do what they're told, consume what they are allowed to consume, demonize those they are told they should demonize. Goebbels would be jealous of the machine the modern world has developed.
> Nostalgia does not guarantee me personal autonomy. Neither does it guarantee me a right to self defense. Nor does it guarantee a free practice of religion, free press, or a right of association.
> Ensuring the survival of a nation state that adheres to the US Constitution will guarantee those things.
This is _precisely_ the uncultured nostalgia I'm suggesting is unworthy of our defenses.
I absolutely support your right to bear arms in your defense. To be free in your religious exercise and associations and speech and press, and in the security of your effects against intrusion.
But the US is not, as you say, a nation state that adheres to the US Constitution in any broad and universal sense.
We have two million people living in cages. Five million more living under supervision of a system of slavery rebranded as carceral rehabilitation. Drugs and medicines and sacraments subject to capricious regulation at best, and often outright prohibition. Fiat currency. Arms which are practical as tools of self-defense unavailable to a quarter of the population.
And I'm not sure that anybody other than (some) white men have _ever_ had these things securely in their civic possession.
So the question is: _can_ a state be the bearer of insurance for these things? Especially now? Why can't we protect each other? Engage in commerce regulated (which is to say, made regular) by the internet alone? I have no idea what the benefits are of the continued existence of the US State, but the downsides are absolutely evident and very real on a daily basis.
> Take a look at China and their authoritarian take on "the internet".
I take your point, absolutely. But doesn't China and the Great Firewall also provide us with an example of failure of authoritarian tendencies as they apply to the internet?
May 35th lives on largely as an internet meme. The official truth may dominate in circles where it can be violently maintained, but those shrink in the face of connectivity, no?
You've hit the nail squarely in your comment. The US is hideously broken. Notice I said "a nation state that adheres to the US constitution". I am not unaware of the failures of the United States, its abuses of power, and the hideous humanity expressed by its elite class (pedophilia, child sacrifice, etc.).
> _can_ a state be the bearer of insurance for these things?
Yes, but it requires the governors of the state to be moral. The peoples of the US have completely failed in passing moral ideologies to their children and grand-children. This moral failure has led to every disintegration of orderly running in the bureaucracy of the US and its states. If the pieces (the people) are immoral, so is the machine.
> And I'm not sure that anybody other than (some) white men have _ever_ had these things securely in their civic possession.
We pretend the opportunity is still to be had. But taxes and inflation steal most of the commoner's earnings. Taxes and inflation see to it that wealth (not necessarily dollars) flows to the elite class. Regulations ensure no one can fight the loss of wealth.
I think people may sometimes forget thag, so far as 95% of the population goes, the internet is still a communication and commerce channels, with endpoints in meatspace. some endpoints are in liberal democracy, some in totalitarian regime. to overthrow the latter you need revolution or nationstate interference.
Policy change-politik-is a function of political will and political efficacy.
the internet is just an information conduit, end of day
> I can absolutely envision - without much difficulty - a megatons-to-megawatts motion swiftly bringing an end to this period in Earth's history.
I can't figure out if you're talking about a nuclear war changing the world, or a nuclear disarmament process that somehow also leads to energy production.
> I can't figure out if you're talking about a nuclear war changing the world, or a nuclear disarmament process that somehow also leads to energy production.
I think you're forgetting that the only thing the internet does is make communication easy. The material realities of where we live, our cultures and belief systems determine how we split into groups like countries.
So while the internet certainly enables us to be more connected as a species, it doesn't eliminate what makes having nations a valuable construct for human societies. As long as our physical environment matters, nations or similar constructs will be around.
I see the Cold War never ended in the US. The communists you're decrying engaged 80% of the German army and were responsible for 7/10s of Nazi deaths on that front.
It wasn't the USSR that dropped two nuclear bombs (one uranium, one plutonium) on Japanese civilian targets when the Japanese were willing to surrender on the sole condition that they keep their emperor (a religious figure)
In the name of "stopping the spread of communism" the US overthrew popularly-elected governments, sent death squads into Latin America, and killed millions of civilians besides the ones they permanently maimed. This is besides the typical "make their economy scream" that kills civilians as well, in a way harder to quantify. Cuba is still the victim of this decades after the failed Bay of Pigs.
We have technically been in a state of emergency since the 50s with the undeclared Korean War, and for what? The permanent wartime economy hasn't benefited anyone besides the military-industrial complex. American imperial designs in Vietnam, Laos, Cambodia, Cuba, Yemen, Syria, etc. were all enabled by this escalation mentality and the crimes against humanity that were committed there are a permanent stain on humanity
What action can actually be taken towards that goal? Your own country can avoid creating these robots. But you can't know that any other country isn't pursuing them. Even if you could, the political capacity to sanction them for it is quite limited. So the most likely outcome of trying to ban them is that every country assumes every other country is still building them in secret, and thus every country still builds them in secret. Eventually, the fact that they are building them in secret becomes public, and ultimately nothing is done about it. Having a world with killer robots in it is bad, but a world where your enemies have killer robots and you don't is far worse. The decision makers will all be very cognizant of that fact.
> But you can't know that any other country isn't pursuing them.
Indeed, what would happen is that if such a treaty was signed, some other countries would develop them anyway, as hidden modes for the use of their otherwise non-autonmous UAVs and UGVs. So any such treaty would essentially be handing the world to countries that cheat -- which would likely include Russia and China.
This kind of reasoning is circular. It justifies any and all escalation on the chance that others are doing the same. Also baked into the assumption is that fire fights fire.
I don't want my state to produce killer robots. Even if other states do. I can only impact my own. It's no secret that the US disproportionately engages in this "warfare" (read: asymmetrical slaughter) and even under a "liberal" head of state (Obama) civilian casualties of our drone program was 90%.
You're also assuming regulating killer robots means a ban, when the international community could restrict deployments or justifiable uses. It's much more realistic even if rogue drones are created.
The decision makers are duly elected representatives of their constituent body and it is beyond time they acted like it. Otherwise we're going to keep having more Vietnams, more Afghanistans, more pointless civilian death.
It's possible to be more cynical about this, but yet to come to a more optimistic conclusion.
If the world's great military powers publicly swear off "killer drones" (let's assume we can meaningfully define such a thing), then they will continue to develop them in secret, but they won't use them in every little border skirmish: every Iraq, Syria, Afghanistan, or Crimea. They'll keep them for the big game.
This has two good effects (assuming further development of killer drones is bad). Firstly, those of us unfortunate enough to be caught up in the horrendous bloody conflicts I callously described as "border skirmishes" won't additionally suffer from killer drone attacks. Secondly, development of the drones and the military doctrine surrounding them will be stymied: you can do a lot in the lab but any technology develops so much faster with the feedback loop that comes from using it in anger.
Moreover, if there's a big war we're all screwed anyway. If that's your focus, you need to argue that drone availablity makes a big war more likely
I'm not convinced that it does - making war cheap makes small war more likely, but all the decision makers know the cost of a big war.
Edit: compare to anti-personnel landmines, or chemical weapons. Both of those see less use because of weapons bans. You might say no, they see less use because the great military powers no longer need them - but you're missing a step. The great powers no longer need them, they decry them as inhumane, they sign bans, and then (almost) everyone stops using them.
Kill-bots change the moral calculus too. As an example, it's one thing to rush a cop on the assumption that human cop won't just gun you down. It's another to rush a programmed kill-bot. The latter case is obviously just suicide, which is morally on the person committing it, not the tool they use to do it.
I'm well aware there are many different ways besides the above one could argue it, but my key point is that automatons change the moral calculus, no matter what other premises you accept.
I think we need an outright ban, but I also understand how workable this strategy is.
Drones have the power to make WWI look like a football match. They could already be in larger use than people realize, it isn't like one would hear about it. There would be nothing to report.
>I'm a citizen of one of the states you mention and I can assure you: amidst all the disagreement and political polarization, there is complete consensus on not wanting a world dominated by killer robots.
If what people "want" is a nice sounding sentence slapped on a roiling pile of inconsistent sub-wants, then good things aren't going to happen.
Sure, nobody wants "a world dominated by killer robots".
Nobody wants "90% civilian casualties" from those killer robots either.
But if people are asked "would you rather Bob down the street get killed fighting in X or let a drone take the risks", what are they going to say? Yes, screw Bob? And that's assuming that the person asked isn't going to be drafted because there are enough volunteers.
If people are asked, "would you rather use drones and kill civilians, or use other methods and kill many more civilians", what are they going to say? Yes, more is better?
Except they aren’t cheap at all (even if they should be) and the defence contractors building them aren’t exactly doing an awesome job for all that money.
The last thing the defence industry wants is for governments to spend less on warfare. If these robots ever get deployed to a battlefield it'll be as something like mechanised landmines or assistive tech for warfighters, not a replacement for existing tech.
The total costs will not go down, but the effectiveness increases substantially until everyone "has to have" the same expensive technology and funding cannot possibly be diminished lest our country becomes vulnerable.
The dreaded "arms race" doing its thing since the dawn of humanity.
The issue is not technology, it is that some states have objectives that're irreconcilable with other states. For example, Russia and China both want to conquer other countries (e.g. Ukraine, Taiwan), something that many countries are against (obviously including the ones threatened with being conquered).
I agree with your analysis that this is a core issue, but something that bothers me with these discussions is:
> If you are a major power with the capability to make 100K+ killer drones
"The major power" is the people that make up the country, which largely don't support this stuff, but there's a handful of decision-makers that can push for this even when the people don't want it. I feel like it's helpful to focus on how it is individual people making these decisions that are going to harm others - not just a result of a country having a certain amount of power/capability. Any one of those people could go public and say that they're not going to let their country go down that path and that they'll discourage others from doing it, but they choose not to for fear of the results.
> "The major power" is the people that make up the country, which largely don't support this stuff
The populations of most countries probably don't want to attack foreign countries, but they absolutely do want to avoid being attacked by foreign countries, and if having scary weapons will do this, most will be for them.
A human can make smart decisions, even if his radio is jammed and he has no connection to base.
What does a "smart" killer robot do, without remote connection, in an unexpected situation? And wars are full of those.
And massive jamming (and hacking attempts as well as sensor confusion) is to be expected. So you either have armies that stop fighting regulary out of savety protocols (how likely is that in a war?).
Or they do not stop fighting and make AI decisions on who to kill.
As far as I am aware, every side knows this and every side is very uncomfortable with this. But it seems every side feels pushed to advance.
And sure, humans make errors, too. But autonomous robots have a tendency to work most of the time - and then unexpectedly make a exponentionally mess.
That's why most robots who can do damage, work in very constraint settings.
But even harmless roombas did make a mess, by swiping dog poo all over the house, while "thinking" it is cleaning.
Swarms of killer bots out of control .. is a scenario no one wants. And military is about control, they do not like things they feel they cannot control. So there will be caution. Hopefully it is enough, to not trigger bigger, potentially nuclear mess.
> How to make sure, it is in the right "kill" area?
If a robot knows where it started from it can use dead reckoning to know roughly where it is. It can also use landmarks it identifies, or celestial navigation.
All of these should give it a reasonably accurate idea of its whereabouts.
> And if you designate a zone as free to kill everything, you might as well nuke the area.
No, because a lot of UGVs will be used as area denial weapons, i.e. as a smart minefield. They might typically stay where they are and monitor than no-one uses the area. Nukes can't do that.
After WWI, the mass horror caused by certain types of weapons like chemical weapons were enough to get all powerful countries to agree we didn't want to see them on the planet.
Hopefully we'll have a chance to recover from the new horrors we are creating too.
Whether you have the capability to make them or not, either the US, Russia, both, or someone else will be HAPPY to sell you a bunch. Which is another reason those with the capability to make them want to keep it, to make the money, maybe even a more significant reason. And countries that _don't_ have the resources to pay for F-35s or air craft carrieres have possibly more to 'gain' here. Killer robots will be a great equalizer in the ability to create a nightmare hellscape, making it accessible on a much cheaper budget.
A more conventional type of attack would be using large numbers cheap cruise missiles. If you can make them for $20k each then $1bn would buy you 50,000 of them, which is probably a good deal more than the number of anti-aircraft missiles your adversary has, and certainly a lot more than their number of fighter aircraft. Therefore firing off a lot in one go would overload their defences and many would get through. If aimed at infrastructure target, they could cripple a country's economy, and how soon after that do people start starving?
I hadn't seen that video, thanks for pointing it out.
Eh, the rest of the world would probably gang up on you at that point, it'd be pretty suicidal.
I think a more likely scenario is killer robots are making the places that state and non-state actors already "get away" with turning into nightmares just that much more nightmarish, and that many more of them. Whether it's the USA in Afghanistan or what goes on in Sudan or Somalia. More and more places like that, which will be more and more nightmarish. Would the USA even have to "give up" on Afghanistan with that tech doing what the drones alone couldn't? But now also the smaller-budget actors like Somalian warlords get their go at it. It's gonna be great.
I believe "slaughterbots" may be able to better evade certain types of air defense (e.g. CIWS), in exchange for maybe being more susceptible to other types of air defense (EMP?).
Additionally, hypothetical slaughterbots would have much lower civilian casualties.
You just put up chicken wire and the slaughterbots get stuck. Most of these weapons have very low tech countermeasures. Where they're effective is when they're not expected. That might work once.
Maybe. The 'slaughterbots' of the future as depicted use explosives to blow holes in buildings. I'd imagine they could arrange multiple in a circle to blow an arbitrarily large hole in fencing or netting.
Also, usually chicken-wire fences just go around what's being protected, not necessarily over it (except for small farm animal enclosures, like actual chicken coops). It's hard for me to imagine entire buildings / neighborhoods / cities with chicken wire covering over the tops of the buildings in a way that couldn't be quickly breached with explosives.
Or drones could unroll detcord in a circle against the fence as well.
If somebody were to pull off a slaughter bot attack there would suddenly be a hundred other countermeasures beyond chicken wire. Slaughterbots as portrayed here are as overhyped as full self driving and the AGI.
>After WWI, the mass horror caused by certain types of weapons like chemical weapons were enough to get all powerful countries to agree we didn't want to see them on the planet.
They were banned because nobody liked them and they weren't tactically useful.
If they were useful they wouldn't have been "banned".
I'm sure the US and Russia will include backdoors into all of the drones they sell which will changes these dynamics a bit. Probably more likely to have proxy wars then rogue wars.
> After WWI, the mass horror caused by certain types of weapons like chemical weapons were enough to get all powerful countries to agree we didn't want to see them on the planet.
Yeah. Kind of. Officially. In primary school books. Chemical weapons (and biological weapons) don't work¹, in addition to producing horrific images and stories.
It was easy to ban them for this combination of reasons.
Killer robots? We don't know yet, but they have the potential of working excellently well. (Although it depends on various reliabilities, the logistics chain required, activation time, percent deployable time, and failure modes. Probably among other things.)
If they work, it will be hard to get them banned. The measure of their effectiveness will be their ability to neutralise the enemy's military capability without destroying economic infrastructure (which you will want to take over and use for yourself. If you don't, there are nuclear weapons for that).
> My quick guess is US, China, Russia, EU (maybe), India as places with enough capacity for this type of next gen battlefield?
Turkey has already developed their own autonomous killer drones (which have in fact killed people, in Libya). So any country with their GDP and level of technology probably could too.
According to Wikipedia, Turkey has the 20th largest economy, and 72nd by per capita GDP. So a lot of countries could develop this technology if they wanted to.
I'd argue against virtually all the points you make BUT you actually miss the main point of this:
this is about making autonomous decisions in the field beyond just local navigation and such. Your arguments hold for non-autonomous drones and autonomous ones to pretty much equal degree. But you don't make any point about why humans shouldn't remain in control of combat decisions.
Consider US vs PRC scenario - naval battle over ocean where theatre area is cordoned off as no-go zone for commercial traffic. UAVs massively out maneuver human pilots. Massive electronic warfare disrupts data links and force remote human out of loop. So UAV falls back to onboard sensors to operate autonomously to finish mission to sink enemy ships. Well preferrable enemy ships that match onboard profiles, but might as well as sink all vessels within a no-go region. No real need for human decision making if goal is already well defined and the environment permissible. Vast percentage of the earth (bodies of water, minimally occupied lands) can easily be turned into no-mans land for UAVs to battle with little risk for collateral damage. It's the same permissible environment for tactical nukes, which is still part of doctrine / planning for anyone that has them.
Because in war humans already move themselves out of those decisions.
Landmines and huge minefields. This is death with no human involvement. If a commander draws a box on a map and says deny the ability of the enemy to cross this box on our flank, does he care particularly whether it's mines (that are a pain to deal with after things quiet down) or if it's killer autonomous drones?
Plant security?
Border patrol?
Think of a rich guy with a private island. Security is wildly expensive, but they are paranoid (check out the security some of the Russian oligarchs run - it's pretty much a mini mercenary force). Would they deploy some automated AI that kills anyone unauthorized (after some warnings?) My guess is yes.
I wonder what group will succumb to genocide by drone first? I figure once you solve the computer vision problem of identifying a group, it's pretty much just an ammunition logistics problem. Imagine The Holocaust without trains and queues. Just a matter of time until another leader needs to scapegoat another group, and next time it will be easier than it's ever been.
It doesn't matter if they regulate it or not. It will happen in secret. It is a foolish hedge, as the nuclear race was.
Fortunately, despite the Deepminds and such, outside limited games we humans still have the upper hand. And us techies, with some effort, can not only defeat killer robots, but we can adversely affect or even control them.
Considering the many woes of humanity, and the inevitable doom we face, I find this an interesting endgame to play. At least now we have something interesting to do while the world burns.
People think of terminators when they hear killer drones, but the cat has been out of the bag for a long time. We've already had Killer Robots for many decades.
It started with icbm's in the 1950s. More sophisticated platforms like the US the US tamohawk missile have been around since the 80s.
I think tamohawks have been able to be programmed with a target hierarchy for a long time, but this is a good point. It seems that The Relevant distinction is automated Friend or Foe identification. Killer drones are not new, but this capability is becoming possible.
The problem with killer robots is that it lowers the political threshold for both starting wars, and for continuing them.
In Europe today, even 2 or 3 dead soldiers is enough to end coalition participation. (Japan's participation in Iraq wasn on the condition of no direct combat and no casualties, so other countries had to babysit the Japanese soldiers.)
The US in WW2 had to invade Japan ASAP before the political will at home faded, which is why unconditional surrender was demanded - there was no way politically to rearm and sail across the Pacific again. Meanwhile, that wasn't a factor for the Axis or Russia.
Japan's behavior in China in WW2 was basically identical to what you see in the "Terminator" movies. Something to think about.
Autonomous weapons don't worry me. There isn't anything they can do that a human controlled weapon cannot do. Most of these autonomous weapons are basically just a different vehicle bringing the payload to the target.
For example, let's say there is a bomb that can be strapped to a drone with a camera on it and guided by AI. That would be easy to evade by just staying out of the open. I'd be much more afraid of my entire city being killed by chemical weapons.
Facial recognition technology will not be a huge game-changer. Most weapons don't even bother to recognize your face to determine if you're a friend or a foe, they just kill you.
Real world autonomous weapons have existed for a long time. The phalanx CIWS is an example of an autonomous weapon designed in the late 60s and it is far from perfect. They tend to just start randomly spraying bullets when they get confused, and they have been involved in friendly fire incidents.
> There isn't anything they can do that a human controlled weapon cannot do
Yes, there is: They don't require human operators, which are a limited ressource. You can only operate so many drones at the same time until you run out of weapon operators / "pilots".
You can't scale human operated weapons in the same way you can autonomous ones, can you? There is nothing stopping you from using hundreds of thousands of cheap, autonomous kamikaze drones at the same time, 24 / 7, if you have the money for it.
> That would be easy to evade by just staying out of the open.
How long can you stay in one building? How long could a drone (or two, with one recharging while the other is flying) circle above it?
Bombing a whole city with chemical weapons, killing countless civilians would cause a major international incident, even with countries not involved in the war / conflict. Killing single individuals with "killer drones" based on AI facial recognition will probably not :/
> You can't scale human operated weapons in the same way you can autonomous ones, can you?
Sure you can. That's what the whole point of an army is. Instead of just having just one tank, have 500 tanks. Sure, scaling does require more manpower, but that's true even with autonomous weapons until someone invents AGI.
> How long can you stay in one building? How long could a drone (or two, with one recharging while the other is flying) circle above it?
Just put a ski mask on. Problem solved. There is just no way that drone swarms with facial recognition technology are ever going to be viable.
Have a listen to this guys lecture on AI and warfare. He explains why we should not fear Skynet instead explains machines capable of selecting targets / people and automatically deciding who should die, their cheep mass production, state of current progress and ways of controlling their use if this needs done.
A CAPTOR mine sits underwater listening to nearby ships or submarines, for weeks to months. If it hears the right sort of ship, it can launch itself autonomously, track it and kill it, without a human in the loop. It's been in service since the 70s.
Only a handful of countries are capable of developing killer robots to maturity and they will end up monopolizing the arms trade market. The tech will proliferate for this reason alone.
If AI is integral to killer drones, then expect a lot more skirmishes going forward to gather data. And realistically there's only a handful of countries / powerblocks who has the impunity to conduct such "research". And when you narrow perquisites down account for synergy with domestic AI industry, that elite club dwindles even smaller. US, PRC, EU if she gets her shit together.
But IMO killer robots along with developments like prompt global strike will trigger dreadnaught tier shift in resetting global power dynamic. Arms race dilemma 101, countries that can, simply can't afford not to pursue them.
I'm sure most countries can get a UAV to drop some munitions on a person. But that's easy mode. Caveat is:
> to maturity
i.e. hightech platforms designed to confront other hightech platforms. Only a handful of country has both the industrial base in terms of AI and MIC to develop drones for capabilities like Anti-air, anti-sub warfare. Most of the world do not have 5th gen planes or modern subs to even train against. Or the host of communication infra, especially space that's needed to get platforms working in field. Only a handful of countries can even contemplate doing that.
That's an interesting idea, but I think you've missed a step in your reasoning. Sadly, a dystopia where powerful elites enslave the majority of the global population could be quite a stable state of affairs, and that dynamic could persist long enough for the civilization to spread throughout the galaxy.
Even if a war between great powers were to occur in such a scenario (which would not be in the interests of the elites who profit from the status quo), I don't think killer robot swarms are any more likely to, or capable of, causing global civilization collapse than nuclear weapons already are.
I think there is something we can glean from what happened with the millions of land mines after conflicts, they were just left there to continue to destroy human lives for decades (probably they can last about a hundred years and still trigger).
Robots that are solar powered and off in remote parts of the world may continue to destroy people when their nation-state owners are gone because of the last MAD command they get.
I don't think that robots "off in remote parts of the world" constitute an existential threat and a "great filter" unless there is some failure mode where they all suddenly get bored and go off to find targets outside their programmed area.
115 comments
[ 2.9 ms ] story [ 184 ms ] threadFrom a realist frame, I don’t think there will be a deal. Narrow national interests are too divergent w/r/t/ capabilities, intentions, and alternatives. Furthermore, verifibility and consequent trust is weak.
Pragmatically it's a lack of higher (multilateral) enforcement why these power plays persist.
Multilateral enforcement becomes yet another tool to take advantage of others that aren't invited.
These are war machines. “Dangerous” per se cannot be a disqualifier.
Automated Safety Enforcement Drones are selective and only issue quick justice to the baddies who stand in the way of freedom! If you can't shame leaders based on their prior agreements civil society doesn't have much else to do.
Really, think about what happens when these tools of warfare return home for civilian use, because if they can, they will.
I guess I’d want to see evidence for this before pursuing an international moratorium everyone will undoubtedly violate the second it’s signed. Remote fire doesn’t seem that much closer than remote kill authorisations.
No shit... people with power will keep a door available for this technology as a tool to use on dissidents.
Like the Geneva Convention, any robot agreement will be ignored by all combatants as soon as it serves their purposes. War cannot be regulated by gentlemen's agreement.
It may not be a gentlemen's agreement, but it's something with which I imagine there are broad consensus across borders.
The USA state will fall eventually. Will it be in 500 years? 100? 20? Nobody knows for sure. But the fall will cause the end (or even substantial disruption) of a tiny portion of the lives of the people living in the regime's claimed territory.
I don't fault your logic; I just think it's out-of-date. The world (especially those parts of it we're discussing here) is really quite different than in the 19th century. Very little of the original impetus for the state (specifically, solving problems of coordination created by the absence of established communications across the land-mass, as a non-trivial portion of The Federalist Papers seek to resolve) remains.
In other words; we'll be fine. We won't merely survive.
So we avert drone war and consume our way off the environment cliff? Or die before but leave the sludge for the future?
I mean, front row seats to heterogeneous droid swarm mayhem sounds fun.
An “I love my job, disposable consumerism, to prop up other peoples grandparents imagined value to society when many in my family got fucked” is not really helping the “but my American civic life story!” crowd. When it was decided in the 80s business taxes were too high, so sad. So is your so called net worth.
Paraphrasing Thomas Jefferson; the dead do not rule the living.
Madisonian philosophy is that we do.
But Jefferson saw servicing generational story of debt as the whole problem in human history.
Adam Smith also agreed; he wrote the powers that be would need insure equality of condition to avoid manipulation of a free labor market (the only market he wrote of).
Somehow we just get Madisonian policy and tall tails of success in a gender and race biased recent history.
If the US wants respect and drone weapons it’s culture should have to prove it’s as capable of behaving responsibly as any other.
Wars generally have a deleterious effect to the life expectancy of the civilian population, even more so in the modern era of mechanized warfare where attacking civilian targets at scale is both practical and practiced (usually not by countries that respect the Geneva conventions, but (a) one cannot guarantee that their opponent will be one of those countries and (b) even those countries have been known to set aside parts of those conventions when they perceive either existential threat or expedience).
No, I don't find it likely at all that most people I know live through a hot war on home soil.
Or nuclear non-proliferation?
Certainly have not been entirely succesful, but I wouldn't want to get rid of them.
As with chemical weapons, they'll still exist, and they'll still be used. But getting the most powerful nations to agree that they are unacceptable and their use has consequences in the international community matters for whether they quickly become pervasive. Vs. allowing the US and Russian defense industries to be in a race to equip the entire world with them as quickly as possible.
[0] https://www.youtube.com/watch?v=9CO6M2HsoIA
Of course, there are grey areas here - certainly there are existing missiles that can be launched without a defined target and programmed to aim for anything that, say, has radar emissions that match known enemy systems.
Drones are the new paradigm of warfare which allows smaller players to play in many situations (like the modern limited local wars) on par with the big ones without attempting to acquire nuclear. No UN talks can prevent those smaller players from development of such capabilities.
My point is that we adapt. It's not worse than chemical weapons and it's preferable to bombing entire cities to achieve your objective. Is it finally dawning on HN that war is hell?
I think the objective should be preventing war and that means making it more difficult to kill not less.
I agree and that's usually the connotation behind the words, "war is hell." Unfortunately, either the entire world agrees to never weaponize drones or we all need to start thinking about weaponizing drones. That's how arms races work.
If wars are going to happen though I think we are better off with drones rather than annihilating entire cities, civilians and soldiers together. It's important to elect leaders who are not cavalier about deploying drones needlessly.
The problem with autonomous weapon platforms (probably swarms) is that it looks like they're very powerful weapons that give their users advantages over existing capabilities. Enforcing a ban on them will be like enforcing a ban on gunpowder with swords and castles.
Which really would have been a good idea. It isn't that all those Europeans had to die to get this current stable state.
What is the drones annual salary? What is their annual housing cost, base allowance costs, VA medical benefits (lifetime)? How upset will your population be if one is "killed". Get the beancounters involved, and killer drones are going to be very tempting. For $75B per year or so (10% of US military budget) what kind of killer drones could I buy and how many?
The cost to deploy one F35 is astronomical. The training costs and lifetime costs for the pilots alone is heavy. The F-35 dedicates MASSIVE payload to life support for its human occupant. What kind of killer drones can you build with none of that mass, none of the gforce limits? None of the redundancy requirements? Do you need an aircraft carrier (thousands of sailors! -> needs destroyers and a battle group). No?
Can we design these to drop from space? Park 50K in space for quick de-orbit and insertion globally? Can you do this with people these days easily?
Going to be a scary world.
My quick guess is US, China, Russia, EU (maybe), India as places with enough capacity for this type of next gen battlefield? Maybe EU will move too slowly, so it'll be US / China / Russia?
Surely the individual decision-makers are, as with all human beings, averse to a world swarming with killer robots.
That states make indefensible decisions with much greater frequency than their individual statesfolk is so obvious at this point; it's time to simply stop tolerating their existence and move more swiftly into the next stage of information humanity.
I'm a citizen of one of the states you mention and I can assure you: amidst all the disagreement and political polarization, there is complete consensus on not wanting a world dominated by killer robots.
That and nukes. It is ok to not want these things, but to survive as a nation state, these things will become necessary developments in order to survive. If GP is correct about some of the capabilities we're likely to see, then any nation state without these will need to begin development in order to ensure survival. Is that an arms race waiting to happen? Yes. Is there a better way? No. The alternative to an escalating arms race is actual war.
We will never be done with power hungry ideologues, we will never be done with Communists whose sole mission is dominion over the whole world, we will never be done with petty despots (looking at you, Putin) who so wish they could play a meaningful role on the world stage. All of these have every reason to develop a new super weapon to tilt the balance of power in their favor.
Iran, for example, has started its centrifuges again. They also have some new ones that spin at 10x the speed of their old ones. They are very close to possessing a nuclear bomb. Up until the end of the Trump presidency they had halted production of uranium and talks were going well. Biden has about the weakest foreign policy of any president in the last half century.
Part of that effective foreign policy is the military might the US has. (Why do we have 5 carrier battle groups? Why does Britain only have one? Why does Russia have a single carrier that's in rather poor repair?)
If killer drones are an effective war-fighting tool (an it appears they are), they are inevitable. What we need to ensure is that they are used as humanely as possible.
Yeah, good point. But I can absolutely envision - without much difficulty - a megatons-to-megawatts motion swiftly bringing an end to this period in Earth's history.
And more generally: maintenance of arsenals of weapons of mass destruction is _very_ costly. Economics of energy production and storage are perhaps a viable mechanism to add pressure in favor of disarmament.
> (Why do we have 5 carrier battle groups? Why does Britain only have one? Why does Russia have a single carrier that's in rather poor repair?)
We can also summarize questions of stagnant geopolitical asymmetry this way:
How long will the internet abide states as nodes of economic and social standing?
Trying to shoehorn a previous world order into an information age society seems like a colossal waste of time in service of nostalgia.
Nostalgia does not guarantee me personal autonomy. Neither does it guarantee me a right to self defense. Nor does it guarantee a free practice of religion, free press, or a right of association.
Ensuring the survival of a nation state that adheres to the US Constitution will guarantee those things.
> How long will the internet abide states as nodes of economic and social standing?
Take a look at China and their authoritarian take on "the internet". "The internet", that is the masses that make it up, do what they're told, consume what they are allowed to consume, demonize those they are told they should demonize. Goebbels would be jealous of the machine the modern world has developed.
> Ensuring the survival of a nation state that adheres to the US Constitution will guarantee those things.
This is _precisely_ the uncultured nostalgia I'm suggesting is unworthy of our defenses.
I absolutely support your right to bear arms in your defense. To be free in your religious exercise and associations and speech and press, and in the security of your effects against intrusion.
But the US is not, as you say, a nation state that adheres to the US Constitution in any broad and universal sense.
We have two million people living in cages. Five million more living under supervision of a system of slavery rebranded as carceral rehabilitation. Drugs and medicines and sacraments subject to capricious regulation at best, and often outright prohibition. Fiat currency. Arms which are practical as tools of self-defense unavailable to a quarter of the population.
And I'm not sure that anybody other than (some) white men have _ever_ had these things securely in their civic possession.
So the question is: _can_ a state be the bearer of insurance for these things? Especially now? Why can't we protect each other? Engage in commerce regulated (which is to say, made regular) by the internet alone? I have no idea what the benefits are of the continued existence of the US State, but the downsides are absolutely evident and very real on a daily basis.
> Take a look at China and their authoritarian take on "the internet".
I take your point, absolutely. But doesn't China and the Great Firewall also provide us with an example of failure of authoritarian tendencies as they apply to the internet?
May 35th lives on largely as an internet meme. The official truth may dominate in circles where it can be violently maintained, but those shrink in the face of connectivity, no?
> _can_ a state be the bearer of insurance for these things?
Yes, but it requires the governors of the state to be moral. The peoples of the US have completely failed in passing moral ideologies to their children and grand-children. This moral failure has led to every disintegration of orderly running in the bureaucracy of the US and its states. If the pieces (the people) are immoral, so is the machine.
> And I'm not sure that anybody other than (some) white men have _ever_ had these things securely in their civic possession.
We pretend the opportunity is still to be had. But taxes and inflation steal most of the commoner's earnings. Taxes and inflation see to it that wealth (not necessarily dollars) flows to the elite class. Regulations ensure no one can fight the loss of wealth.
Policy change-politik-is a function of political will and political efficacy.
the internet is just an information conduit, end of day
I can't figure out if you're talking about a nuclear war changing the world, or a nuclear disarmament process that somehow also leads to energy production.
My apologies - the latter.
So while the internet certainly enables us to be more connected as a species, it doesn't eliminate what makes having nations a valuable construct for human societies. As long as our physical environment matters, nations or similar constructs will be around.
Yes, exactly.
> If killer drones are an effective war-fighting tool (an it appears they are), they are inevitable.
I have come to the same conclusion.
It wasn't the USSR that dropped two nuclear bombs (one uranium, one plutonium) on Japanese civilian targets when the Japanese were willing to surrender on the sole condition that they keep their emperor (a religious figure)
In the name of "stopping the spread of communism" the US overthrew popularly-elected governments, sent death squads into Latin America, and killed millions of civilians besides the ones they permanently maimed. This is besides the typical "make their economy scream" that kills civilians as well, in a way harder to quantify. Cuba is still the victim of this decades after the failed Bay of Pigs.
We have technically been in a state of emergency since the 50s with the undeclared Korean War, and for what? The permanent wartime economy hasn't benefited anyone besides the military-industrial complex. American imperial designs in Vietnam, Laos, Cambodia, Cuba, Yemen, Syria, etc. were all enabled by this escalation mentality and the crimes against humanity that were committed there are a permanent stain on humanity
Heh, that's quite the narrative.
https://en.wikipedia.org/wiki/Ky%C5%ABj%C5%8D_incident
Indeed, what would happen is that if such a treaty was signed, some other countries would develop them anyway, as hidden modes for the use of their otherwise non-autonmous UAVs and UGVs. So any such treaty would essentially be handing the world to countries that cheat -- which would likely include Russia and China.
I don't want my state to produce killer robots. Even if other states do. I can only impact my own. It's no secret that the US disproportionately engages in this "warfare" (read: asymmetrical slaughter) and even under a "liberal" head of state (Obama) civilian casualties of our drone program was 90%.
You're also assuming regulating killer robots means a ban, when the international community could restrict deployments or justifiable uses. It's much more realistic even if rogue drones are created.
The decision makers are duly elected representatives of their constituent body and it is beyond time they acted like it. Otherwise we're going to keep having more Vietnams, more Afghanistans, more pointless civilian death.
If the world's great military powers publicly swear off "killer drones" (let's assume we can meaningfully define such a thing), then they will continue to develop them in secret, but they won't use them in every little border skirmish: every Iraq, Syria, Afghanistan, or Crimea. They'll keep them for the big game.
This has two good effects (assuming further development of killer drones is bad). Firstly, those of us unfortunate enough to be caught up in the horrendous bloody conflicts I callously described as "border skirmishes" won't additionally suffer from killer drone attacks. Secondly, development of the drones and the military doctrine surrounding them will be stymied: you can do a lot in the lab but any technology develops so much faster with the feedback loop that comes from using it in anger.
Moreover, if there's a big war we're all screwed anyway. If that's your focus, you need to argue that drone availablity makes a big war more likely I'm not convinced that it does - making war cheap makes small war more likely, but all the decision makers know the cost of a big war.
Edit: compare to anti-personnel landmines, or chemical weapons. Both of those see less use because of weapons bans. You might say no, they see less use because the great military powers no longer need them - but you're missing a step. The great powers no longer need them, they decry them as inhumane, they sign bans, and then (almost) everyone stops using them.
See for example Bret Devereux: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
I'm well aware there are many different ways besides the above one could argue it, but my key point is that automatons change the moral calculus, no matter what other premises you accept.
Drones have the power to make WWI look like a football match. They could already be in larger use than people realize, it isn't like one would hear about it. There would be nothing to report.
The history of the use of land and sea mines in warfare suggests to me that this is not a universal we can assume.
If what people "want" is a nice sounding sentence slapped on a roiling pile of inconsistent sub-wants, then good things aren't going to happen.
Sure, nobody wants "a world dominated by killer robots".
Nobody wants "90% civilian casualties" from those killer robots either.
But if people are asked "would you rather Bob down the street get killed fighting in X or let a drone take the risks", what are they going to say? Yes, screw Bob? And that's assuming that the person asked isn't going to be drafted because there are enough volunteers.
If people are asked, "would you rather use drones and kill civilians, or use other methods and kill many more civilians", what are they going to say? Yes, more is better?
Remember this? https://www.wsj.com/articles/SB126102247889095011
The dreaded "arms race" doing its thing since the dawn of humanity.
> If you are a major power with the capability to make 100K+ killer drones
"The major power" is the people that make up the country, which largely don't support this stuff, but there's a handful of decision-makers that can push for this even when the people don't want it. I feel like it's helpful to focus on how it is individual people making these decisions that are going to harm others - not just a result of a country having a certain amount of power/capability. Any one of those people could go public and say that they're not going to let their country go down that path and that they'll discourage others from doing it, but they choose not to for fear of the results.
The populations of most countries probably don't want to attack foreign countries, but they absolutely do want to avoid being attacked by foreign countries, and if having scary weapons will do this, most will be for them.
What does a "smart" killer robot do, without remote connection, in an unexpected situation? And wars are full of those.
And massive jamming (and hacking attempts as well as sensor confusion) is to be expected. So you either have armies that stop fighting regulary out of savety protocols (how likely is that in a war?). Or they do not stop fighting and make AI decisions on who to kill.
As far as I am aware, every side knows this and every side is very uncomfortable with this. But it seems every side feels pushed to advance.
And sure, humans make errors, too. But autonomous robots have a tendency to work most of the time - and then unexpectedly make a exponentionally mess.
That's why most robots who can do damage, work in very constraint settings.
But even harmless roombas did make a mess, by swiping dog poo all over the house, while "thinking" it is cleaning.
Swarms of killer bots out of control .. is a scenario no one wants. And military is about control, they do not like things they feel they cannot control. So there will be caution. Hopefully it is enough, to not trigger bigger, potentially nuclear mess.
Possibly kill anything that looks like an enemy, then attempt to return to base or self-destruct.
I expect killer robots will be programmed with geographic areas within which they are allowed to kill things.
What about sensor jamming or confusing? How to make sure, it is in the right "kill" area?
And if you designate a zone as free to kill everything, you might as well nuke the area.
This will no doubt happen.
> How to make sure, it is in the right "kill" area?
If a robot knows where it started from it can use dead reckoning to know roughly where it is. It can also use landmarks it identifies, or celestial navigation.
All of these should give it a reasonably accurate idea of its whereabouts.
> And if you designate a zone as free to kill everything, you might as well nuke the area.
No, because a lot of UGVs will be used as area denial weapons, i.e. as a smart minefield. They might typically stay where they are and monitor than no-one uses the area. Nukes can't do that.
Smart as in kills anything that moves, sure.
"All of these should give it a reasonably accurate idea of its whereabouts."
But if you add high speeds and bad weather to the scenario, together with highly populated areas, reasonable accurate becomes a relative thing.
Unless the drones as a consequence mostly fight themself in remote areas.
Might be better than humans fighting in urban areas, but reality most likely will be robots vs. bad equipped humans in urban areas.
Hopefully we'll have a chance to recover from the new horrors we are creating too.
Whether you have the capability to make them or not, either the US, Russia, both, or someone else will be HAPPY to sell you a bunch. Which is another reason those with the capability to make them want to keep it, to make the money, maybe even a more significant reason. And countries that _don't_ have the resources to pay for F-35s or air craft carrieres have possibly more to 'gain' here. Killer robots will be a great equalizer in the ability to create a nightmare hellscape, making it accessible on a much cheaper budget.
I imagine most people here have already seen the Slaughterbots video: https://www.youtube.com/watch?v=9CO6M2HsoIA
A more conventional type of attack would be using large numbers cheap cruise missiles. If you can make them for $20k each then $1bn would buy you 50,000 of them, which is probably a good deal more than the number of anti-aircraft missiles your adversary has, and certainly a lot more than their number of fighter aircraft. Therefore firing off a lot in one go would overload their defences and many would get through. If aimed at infrastructure target, they could cripple a country's economy, and how soon after that do people start starving?
Eh, the rest of the world would probably gang up on you at that point, it'd be pretty suicidal.
I think a more likely scenario is killer robots are making the places that state and non-state actors already "get away" with turning into nightmares just that much more nightmarish, and that many more of them. Whether it's the USA in Afghanistan or what goes on in Sudan or Somalia. More and more places like that, which will be more and more nightmarish. Would the USA even have to "give up" on Afghanistan with that tech doing what the drones alone couldn't? But now also the smaller-budget actors like Somalian warlords get their go at it. It's gonna be great.
Additionally, hypothetical slaughterbots would have much lower civilian casualties.
https://spectrum.ieee.org/why-you-shouldnt-fear-slaughterbot...
Also, usually chicken-wire fences just go around what's being protected, not necessarily over it (except for small farm animal enclosures, like actual chicken coops). It's hard for me to imagine entire buildings / neighborhoods / cities with chicken wire covering over the tops of the buildings in a way that couldn't be quickly breached with explosives.
Or drones could unroll detcord in a circle against the fence as well.
They were banned because nobody liked them and they weren't tactically useful.
If they were useful they wouldn't have been "banned".
[1] https://en.wikipedia.org/wiki/UK_arms_export
Yeah. Kind of. Officially. In primary school books. Chemical weapons (and biological weapons) don't work¹, in addition to producing horrific images and stories.
It was easy to ban them for this combination of reasons.
Killer robots? We don't know yet, but they have the potential of working excellently well. (Although it depends on various reliabilities, the logistics chain required, activation time, percent deployable time, and failure modes. Probably among other things.)
If they work, it will be hard to get them banned. The measure of their effectiveness will be their ability to neutralise the enemy's military capability without destroying economic infrastructure (which you will want to take over and use for yourself. If you don't, there are nuclear weapons for that).
1. https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
Turkey has already developed their own autonomous killer drones (which have in fact killed people, in Libya). So any country with their GDP and level of technology probably could too.
According to Wikipedia, Turkey has the 20th largest economy, and 72nd by per capita GDP. So a lot of countries could develop this technology if they wanted to.
this is about making autonomous decisions in the field beyond just local navigation and such. Your arguments hold for non-autonomous drones and autonomous ones to pretty much equal degree. But you don't make any point about why humans shouldn't remain in control of combat decisions.
Landmines and huge minefields. This is death with no human involvement. If a commander draws a box on a map and says deny the ability of the enemy to cross this box on our flank, does he care particularly whether it's mines (that are a pain to deal with after things quiet down) or if it's killer autonomous drones?
Plant security?
Border patrol?
Think of a rich guy with a private island. Security is wildly expensive, but they are paranoid (check out the security some of the Russian oligarchs run - it's pretty much a mini mercenary force). Would they deploy some automated AI that kills anyone unauthorized (after some warnings?) My guess is yes.
Fortunately, despite the Deepminds and such, outside limited games we humans still have the upper hand. And us techies, with some effort, can not only defeat killer robots, but we can adversely affect or even control them.
Considering the many woes of humanity, and the inevitable doom we face, I find this an interesting endgame to play. At least now we have something interesting to do while the world burns.
It started with icbm's in the 1950s. More sophisticated platforms like the US the US tamohawk missile have been around since the 80s.
In Europe today, even 2 or 3 dead soldiers is enough to end coalition participation. (Japan's participation in Iraq wasn on the condition of no direct combat and no casualties, so other countries had to babysit the Japanese soldiers.)
The US in WW2 had to invade Japan ASAP before the political will at home faded, which is why unconditional surrender was demanded - there was no way politically to rearm and sail across the Pacific again. Meanwhile, that wasn't a factor for the Axis or Russia.
Japan's behavior in China in WW2 was basically identical to what you see in the "Terminator" movies. Something to think about.
For example, let's say there is a bomb that can be strapped to a drone with a camera on it and guided by AI. That would be easy to evade by just staying out of the open. I'd be much more afraid of my entire city being killed by chemical weapons.
Facial recognition technology will not be a huge game-changer. Most weapons don't even bother to recognize your face to determine if you're a friend or a foe, they just kill you.
Real world autonomous weapons have existed for a long time. The phalanx CIWS is an example of an autonomous weapon designed in the late 60s and it is far from perfect. They tend to just start randomly spraying bullets when they get confused, and they have been involved in friendly fire incidents.
Yes, there is: They don't require human operators, which are a limited ressource. You can only operate so many drones at the same time until you run out of weapon operators / "pilots".
You can't scale human operated weapons in the same way you can autonomous ones, can you? There is nothing stopping you from using hundreds of thousands of cheap, autonomous kamikaze drones at the same time, 24 / 7, if you have the money for it.
> That would be easy to evade by just staying out of the open.
How long can you stay in one building? How long could a drone (or two, with one recharging while the other is flying) circle above it?
Bombing a whole city with chemical weapons, killing countless civilians would cause a major international incident, even with countries not involved in the war / conflict. Killing single individuals with "killer drones" based on AI facial recognition will probably not :/
Sure you can. That's what the whole point of an army is. Instead of just having just one tank, have 500 tanks. Sure, scaling does require more manpower, but that's true even with autonomous weapons until someone invents AGI.
> How long can you stay in one building? How long could a drone (or two, with one recharging while the other is flying) circle above it?
Just put a ski mask on. Problem solved. There is just no way that drone swarms with facial recognition technology are ever going to be viable.
For me it's not so much about what they can/cannot do, but what they will/will not do.
https://www.bbc.co.uk/programmes/m00127t9
It’s really up to date to as he is also delivering a message to those at the UN talks today
A CAPTOR mine sits underwater listening to nearby ships or submarines, for weeks to months. If it hears the right sort of ship, it can launch itself autonomously, track it and kill it, without a human in the loop. It's been in service since the 70s.
If AI is integral to killer drones, then expect a lot more skirmishes going forward to gather data. And realistically there's only a handful of countries / powerblocks who has the impunity to conduct such "research". And when you narrow perquisites down account for synergy with domestic AI industry, that elite club dwindles even smaller. US, PRC, EU if she gets her shit together.
But IMO killer robots along with developments like prompt global strike will trigger dreadnaught tier shift in resetting global power dynamic. Arms race dilemma 101, countries that can, simply can't afford not to pursue them.
Every nation on earth has the capability.
> to maturity
i.e. hightech platforms designed to confront other hightech platforms. Only a handful of country has both the industrial base in terms of AI and MIC to develop drones for capabilities like Anti-air, anti-sub warfare. Most of the world do not have 5th gen planes or modern subs to even train against. Or the host of communication infra, especially space that's needed to get platforms working in field. Only a handful of countries can even contemplate doing that.
If not, the first casualty of the next war in the Middle East will be the closest Department of Algebra, because the name sounds Arabic.
I wish I was joking. The potential for bad decisions in such autonomous systems is tremendous.
"US Defence Secretary William Cohen has confirmed that an out-of-date map was used in the attack which destroyed the Chinese embassy in Belgrade."
http://news.bbc.co.uk/1/hi/world/europe/340735.stm
It seems entirely inevitable that these machines will be everywhere policing the masses for the powerful few.
Even if a war between great powers were to occur in such a scenario (which would not be in the interests of the elites who profit from the status quo), I don't think killer robot swarms are any more likely to, or capable of, causing global civilization collapse than nuclear weapons already are.
Robots that are solar powered and off in remote parts of the world may continue to destroy people when their nation-state owners are gone because of the last MAD command they get.