I've never understood the commonly expressed idea that if the end of the world seems inevitable that it is therefore logical to work towards it.
Why not...just not? And see what happens? We're all guaranteed to die anyway, so what is there to lose? It's not rational to bet on the end of the world in any possible situation.
I'm having similar thoughts a lot recently, which are far from original to me, yet I am disturbed by all the people who apparently don't have them.
Like, the richest guy in the world cares about union busting and tax breaks at the moment, but why? I literally can't imagine anything he can gain from being a bastard at this point, and so I can't even get viscerally angry. It's more like contemplating the mysteries of a tornado. What the hell makes it go and what would stop it?
So far, anyway. The more you keep advancing ways to end the world, and make them cheaper and easier, the more risk you create that they fall in the hands of someone whose actual goal is to end the world rather than deterrence.
Except that they don't see it as the end of the world but rather a guaranteed cost when someone considers an invasion.
And as far as I know, autonomous weapons don't inherently involve scorched earth scenarios. Perhaps in the far future the worst war we have will be each side attacking the other's autonomous weapons and then calling a ceasefire because humans are too costly to be in the frontlines.
I have the opposite viewpoint, in that I think the one real leap autonomous weapons represent is that they can be programmed to treat human and any other life as even less valuable than the most depraved psychopath would, and therefore can terminate any amount in a very short time without asking for a reason.
I would agree that these weapons should be out of the hands of the average civilian gunowner and put in the hands of say, army commanders. But then you already have the civilian gunowners who dont go through regular psyche tests no?
When people aren't dying in the front lines because it's just robots, there's no reason for a ceasefire. There won't be a human cost to fighting, only for running out of money and losing.
In some manner, pursuing a strategy that allows you to end the world allows you to stave off ending the world.
Think of it through the lens of an iterated prisoner's dilemma: if I suddenly develop deathbots and you don't have them, it changes the payoffs for defection/cooperation which could cause me to change strategies and kill you.
I would recommend Axelrod's book "The Evolution of Cooperation" for ideas surrounding this.
Doesn't work. All it takes is one power hungry dictator to screw it all that up. Peace requires either a stalemate or a benevolent(this is not the right word, someone please help) overpowered force to threaten others to stay in line.
He probably meant "a power not interested in war for war's sake, content with the territorial status quo and preferring diplomacy to use of force, though not beyond use of force if deemed necessary".
Expressing this quality in a single word is a challenge.
Consider the statement "Autonomous weapons are inevitable" carefully. What it means is, any state actor can potentially invest in tech to develop these type of weapons and succeed in a few years. In other words autonomous weapons are within striking distance of technological progress (as opposed to, say, human landing on Mars.)
With the above information, what would you do, if you were a state actor of an advanced country? Surely, (1) you either want to develop these yourself, or (2) ensure that nobody else develops them. (2) can be achieved by international treaties, but the question is whether it can be enforced. If state actors don't trust option (2), then the only option left is (1).
There are more than two options. Some other options include building defenses against such weapons or actively working to subvert the other side's building process.
Okay - Without understanding a specific tech, would it be possible to defend against it? And subversion may not be possible against certain countries. I would rather have these type of weapons developed by countries ruled by democracy.
Autonomous weapons do not translate to an inevitable end of the world any more than all the previous weapon systems constructed by humans. Warfare and diplomacy adapt to new developments.
At the very least weapons are controllable; they have a kill switch. Things like SARS-Cov-2 do not have a kill switch.
> However, only around 30 countries currently support the ban.
While it's completely pragmatic to pursue autonomous weapons due to peer competition, I wonder if it will paradoxically erode US warfighting edge when effective algorithms becomes ubiquitous. Why Arab Lose Wars -> Will Arabs Lose Wars. Incompetent forces previously limited by poor institutions and traditions that would otherwise take generations to cultivate can be improved closer to global parity by importing competent murder bots. Wonder how monkey models of exported AI weapons will have functions limited.
>Provided their use is authorized by a human commander or operator
>can be improved closer to global parity by importing competent murder bots
I remember various people writing about the ebbs and flows of centralized oppressive government and attributing it to the type of weapon that's prevalent and the support system it requires. George Orwell writing just after WWII was very apprehensive about more than just the atomic bomb per se, but by the idea that the best weapons available would be only available through massive industrial projects of the largest governments and therefore all the greatest despotisms would be able to survive indefinitely.
Your comment also reminds me of reading somewhere a long time ago how the US having a decent phone system left it with a lot of copper infrastructure that held things back, whereas some countries that have been much poorer in the past went directly to cellular - thus in general wealth implies early adoption and possibly too much investment in immature tech.
I think competent AI weapons will dramatically reduce barriers to projecting and holding power by smaller ruling cohorts, less dependence on training massive armed forces and retaining their loyalty. Both extremely difficult and expensive institution problems versus importing a bunch of DRM murder bots a medium sized country can't manufacture, or just receiving some in form of aid from a larger power in exchange for client status. Don't piss off benefactors or DRM transfers to more compliant political faction.
>too much investment in immature tech
Or research into paradigms shifting tech negates moat of old advantages. Russians good at antiair because US would not want to invent tech that would erode US air supremacy. Kind of how UK lost naval gap due to dreadnaught reset. I can imagine AI air planes undermining US air / naval superiority balance. Lots of obsolete hardware and training if a fleet of drones become smart enough to penetrate a carrier group. Perhaps it may even be sensible not to pursue / start arms race if it means undermining decades of expensive acquisitions and expertise. Or at least have enough security to ensure adversaries don't acquire kryptonite of your own making.
Competent murder bots won't choose where to be deployed, for what reasons, etc. As long as the Arab countries are outposts of the US and do whatever the US wants blindly, they'll fail just like the Iraqi forces bolstered by the US failed. It's not because the US is bad or the Arabs who are their allies are bad. It's because Arabs who work for the US are essentially mercenaries, they don't care what the result is as long as they get paid. Just as Rome fell partly because of its heavy reliance on mercenaries, so too will these fail.
Many Arab forces are highly competent (and feared as a result) but they all have in common that they are not simply lavishly compensated by the US to do things they don't want to do, rather they are fervent recruits bound by some shared ideology and goal. I'm thinking of the Shia PMUs, the Taliban (maybe not considered Arab), Hezbollah, Ansarullah/Houthis, etc. These people are the opposites of mercenaries, and they often win outsized victories as a result.
The Russian and Chinese governments care little about niceties such as ethics and morality, which can be an advantage.
I foresee the development of genetically enhanced super soldiers. While Western democracies will be embroiled in unproductive social debates, which will lead to "commissions" and "investigations", China and Russia will press ahead with armies of clone troopers.
*If clone troopers can be bred (1) at scale (2) reliably (3) and affordably.
China voted for the ban of use autonomous weapons. Russia indicated that its willing to sign treaties against their use.
So far, the only country that deployed an autonomous weapons system is South Korea, a US ally and Israel which is also a US ally.
Russia has only ever deployed defensive autonomous system and fail-deadly nuclear weapons system which even remotely fit the definition.
China has presicely nothing.
Neither China nor Russia is breeding "clone troopers".
From where I'm sitting at, this really seems like barely more than baiting fear using The Foreign Inhuman.
To note, neither Russia nor China have anything to gain relatively from autonomous weapons or "clone troopers" - Russia is at a technological disadvantage in those domains, while China has no plans nor incentives to fight a large scale land war, in which case they would already have more soldiers than they can use, and technoligically is at best matched in autonomous systems with its main adversaries.
where is your source. How do you know that china is NOT doing that? You have no idea. Just as we have no idea that the US was building a human tracking system since 9/11 and kept it secret during the wars. Now we know.
I mean, if the first nation to orbit a teapot around the sun would instantly become a nearly-unstoppable military hegemon, it would be pretty reasonable to assume that there were a secret teapot-orbiting project going on in multiple countries.
There is no such advantage to building autonomous weapons, not at this stage. Where China is so far, their limitation isn't finding the staff to control weapons, and they don't face a situation where expandability is the main issue - their issue is mainly finishing catching up militarily.
So the issue is really your premises about it making you "nearly unstoppable", it leads to no such thing. The advantage is much slimmer than you might think, especially right now. As it is, unmanned but non-autonomous weapons provide 99% of the functionality.
Additionally, neither the US, Israel, the UK, or Russia are making their autonomous weapons programs secret.
Proof? China, Russia and USA have all lied before about weapons many times. Of course USA should sign the treaty and go on developing the weapon in secret.
Proof of what? I can't possibly prove a negative. The facts is that China hasn't deployed any such weapons, while US allies have (deployed weapons are well documented especially when as visible as aircraft and other mass produced vehicles).
As for votes and public statements on treaties, that's all public information.
There’s no stopping this. It’s getting easier and smaller to build autonomous robots. Even now you can buy a cheap drone under $100 strap a bomb and a phone to it and have it fly to a destination and detonate.
Ignoring calls to ban autonomous weapons is unfortunately the only rational course of action. Nuclear development could be prevented because it's easy to detect, thus making the NPT workable. Even cluster bomb bans are workable because they're single-use. It's very difficult to detect development of autonomous weapons in a way that would stick in the international he-said-she-said game, until they're deployed in the field, by which point it's already too late. The costs of a hostile power developing a decisive advantage is too high.
It's unfortunate that this puts us on a possible path to self-replicating AI war bots and an endless war between two robot armies with nobody left alive at the helm, but the alternative is asking people to leave themselves vulnerable to invasion and subjugation, which simply won't fly.
I read Carl Sagan praising spy satellites for stopping the escalating "but the other side is...." and later, UN inspectors diffusing a potential US-Iran war. The future international industrial inspection (III is the name!) system to prevent AI weapons could finally cut off the profits of unethical/unsustainable crops/mined products, slave labor, illegal fishing, gunrunning etc.
You'd think of all people in the world, Eric Schmidt would be well aware of how well intentioned powerful technology can end up having unforeseen repercussions
Instead of going into these abstract discussions about arms-races and how everything is inevitable, can someone give me a practical example of where simply not using autonomous weapons has disastrous implications for national defense?
Because as I see it the primary domains where these weapons seem to excel is drone-warfare, border control, surveillance and so on, not necessarily national defense.
These technologies seem more like a backdoor to eliminate human ethical concerns from decision-making and empower domestic policing (which is where military technology tends to go), rather than defending the country. What's a concrete example where, even when China or Russia do not honor a ban, the US could not simply strike with a human in the loop?
Democratic societies who value human rights already forego the use of countless of military tactics they could potentially employ. It hasn't made any of them defenseless.
Drone warfare and border control ARE national defense.
That aside, if China or Russia gets AI robot guns it’s gonna be like boomers playing against kids with aimbot in counter strike. The US will get destroyed.
Just to add one small note here: A drone is, in academic literature on security, not a autonomous weapon or robot.
Drones are assumed to be remote controlled, therefore retain a human decision maker at all times. Robots would make such decisions themselves.
While drones come with their own set of problems (usual suspects are transparency, deployment regulations, sovereignty, decision making without skin in the game, etc.) they are distinct from the set of problems robots / "autonomous" weapons introduce. (Like distinction, proportionality, transparency & responsibility)
I am in agreement with you in regards to the dangers autonomous weapons pose for domestic use, but unfortunately drone-warfare is a national defense issue and already is impacting how wars are fought.
Just to be precise here, I'm not advocating ignoring all modern technologies like drones altogether. The concern here is autonomous weaponry in particular.
I'm against taking the human out of the loop and leaving the decision and responsibility to kill to a machine. I can think of many situations where a drone is a necessary technology to win a war but of far fewer where a human confirmation cannot be afforded.
We already use land mines, which are automated weapons. The US is one of the few countries not to sign on to the 1997 Mine Ban Treaty, and its self-imposed restrictions were lifted a year ago. As far as I know they haven't been restored yet.
It seems like humanity is stuck in an ever-increasing consideration of Game Theory. It seems this conceptual model will always trend towards the direction of puffing up bigger, and bigger chests.
Is there a possibility we could be having it all wrong? The potential to assume non-confrontation instead of environmental risk?
58 comments
[ 3.7 ms ] story [ 123 ms ] threadWhy not...just not? And see what happens? We're all guaranteed to die anyway, so what is there to lose? It's not rational to bet on the end of the world in any possible situation.
I'm having similar thoughts a lot recently, which are far from original to me, yet I am disturbed by all the people who apparently don't have them.
Like, the richest guy in the world cares about union busting and tax breaks at the moment, but why? I literally can't imagine anything he can gain from being a bastard at this point, and so I can't even get viscerally angry. It's more like contemplating the mysteries of a tornado. What the hell makes it go and what would stop it?
Because the consequence being the end of the world rather than just a large part of it is what keeps the peace.
And as far as I know, autonomous weapons don't inherently involve scorched earth scenarios. Perhaps in the far future the worst war we have will be each side attacking the other's autonomous weapons and then calling a ceasefire because humans are too costly to be in the frontlines.
Think of it through the lens of an iterated prisoner's dilemma: if I suddenly develop deathbots and you don't have them, it changes the payoffs for defection/cooperation which could cause me to change strategies and kill you.
I would recommend Axelrod's book "The Evolution of Cooperation" for ideas surrounding this.
Perhaps you meant dispassionate?
Expressing this quality in a single word is a challenge.
With the above information, what would you do, if you were a state actor of an advanced country? Surely, (1) you either want to develop these yourself, or (2) ensure that nobody else develops them. (2) can be achieved by international treaties, but the question is whether it can be enforced. If state actors don't trust option (2), then the only option left is (1).
At the very least weapons are controllable; they have a kill switch. Things like SARS-Cov-2 do not have a kill switch.
The actual title provides key context.
While it's completely pragmatic to pursue autonomous weapons due to peer competition, I wonder if it will paradoxically erode US warfighting edge when effective algorithms becomes ubiquitous. Why Arab Lose Wars -> Will Arabs Lose Wars. Incompetent forces previously limited by poor institutions and traditions that would otherwise take generations to cultivate can be improved closer to global parity by importing competent murder bots. Wonder how monkey models of exported AI weapons will have functions limited.
>Provided their use is authorized by a human commander or operator
https://static.wikia.nocookie.net/simpsons/images/d/da/Dimes...
I remember various people writing about the ebbs and flows of centralized oppressive government and attributing it to the type of weapon that's prevalent and the support system it requires. George Orwell writing just after WWII was very apprehensive about more than just the atomic bomb per se, but by the idea that the best weapons available would be only available through massive industrial projects of the largest governments and therefore all the greatest despotisms would be able to survive indefinitely.
Your comment also reminds me of reading somewhere a long time ago how the US having a decent phone system left it with a lot of copper infrastructure that held things back, whereas some countries that have been much poorer in the past went directly to cellular - thus in general wealth implies early adoption and possibly too much investment in immature tech.
>too much investment in immature tech
Or research into paradigms shifting tech negates moat of old advantages. Russians good at antiair because US would not want to invent tech that would erode US air supremacy. Kind of how UK lost naval gap due to dreadnaught reset. I can imagine AI air planes undermining US air / naval superiority balance. Lots of obsolete hardware and training if a fleet of drones become smart enough to penetrate a carrier group. Perhaps it may even be sensible not to pursue / start arms race if it means undermining decades of expensive acquisitions and expertise. Or at least have enough security to ensure adversaries don't acquire kryptonite of your own making.
Many Arab forces are highly competent (and feared as a result) but they all have in common that they are not simply lavishly compensated by the US to do things they don't want to do, rather they are fervent recruits bound by some shared ideology and goal. I'm thinking of the Shia PMUs, the Taliban (maybe not considered Arab), Hezbollah, Ansarullah/Houthis, etc. These people are the opposites of mercenaries, and they often win outsized victories as a result.
I foresee the development of genetically enhanced super soldiers. While Western democracies will be embroiled in unproductive social debates, which will lead to "commissions" and "investigations", China and Russia will press ahead with armies of clone troopers.
*If clone troopers can be bred (1) at scale (2) reliably (3) and affordably.
So far, the only country that deployed an autonomous weapons system is South Korea, a US ally and Israel which is also a US ally.
Russia has only ever deployed defensive autonomous system and fail-deadly nuclear weapons system which even remotely fit the definition.
China has presicely nothing.
Neither China nor Russia is breeding "clone troopers".
From where I'm sitting at, this really seems like barely more than baiting fear using The Foreign Inhuman.
To note, neither Russia nor China have anything to gain relatively from autonomous weapons or "clone troopers" - Russia is at a technological disadvantage in those domains, while China has no plans nor incentives to fight a large scale land war, in which case they would already have more soldiers than they can use, and technoligically is at best matched in autonomous systems with its main adversaries.
So the issue is really your premises about it making you "nearly unstoppable", it leads to no such thing. The advantage is much slimmer than you might think, especially right now. As it is, unmanned but non-autonomous weapons provide 99% of the functionality.
Additionally, neither the US, Israel, the UK, or Russia are making their autonomous weapons programs secret.
Beyond that, the US and Russia don't seem to be hiding their work on it, so why would China, which has even less of an incentive, be hiding it?
As for votes and public statements on treaties, that's all public information.
Almost every country in the world has unmanned weapons. That's not the issue. The issue is autonomous weapons.
It's unfortunate that this puts us on a possible path to self-replicating AI war bots and an endless war between two robot armies with nobody left alive at the helm, but the alternative is asking people to leave themselves vulnerable to invasion and subjugation, which simply won't fly.
Because as I see it the primary domains where these weapons seem to excel is drone-warfare, border control, surveillance and so on, not necessarily national defense.
These technologies seem more like a backdoor to eliminate human ethical concerns from decision-making and empower domestic policing (which is where military technology tends to go), rather than defending the country. What's a concrete example where, even when China or Russia do not honor a ban, the US could not simply strike with a human in the loop?
Democratic societies who value human rights already forego the use of countless of military tactics they could potentially employ. It hasn't made any of them defenseless.
That aside, if China or Russia gets AI robot guns it’s gonna be like boomers playing against kids with aimbot in counter strike. The US will get destroyed.
Drones are assumed to be remote controlled, therefore retain a human decision maker at all times. Robots would make such decisions themselves.
While drones come with their own set of problems (usual suspects are transparency, deployment regulations, sovereignty, decision making without skin in the game, etc.) they are distinct from the set of problems robots / "autonomous" weapons introduce. (Like distinction, proportionality, transparency & responsibility)
https://www.forbes.com/sites/pauliddon/2021/03/25/the-last-a...
I'm against taking the human out of the loop and leaving the decision and responsibility to kill to a machine. I can think of many situations where a drone is a necessary technology to win a war but of far fewer where a human confirmation cannot be afforded.
Is there a possibility we could be having it all wrong? The potential to assume non-confrontation instead of environmental risk?
With our current programming, I guess not.