58 comments

[ 0.25 ms ] story [ 117 ms ] thread
You can look on YouTube on weaponized version : https://www.youtube.com/watch?v=mTpGhAx-NbY
Watching this I can see a UN Convention on Autonomous Weapons is inevitable.
All wars should be autonomous vehicles vs autonomous vehicle. A pile of scrap is better than a pile of bodies.
and the next step from there is simply to simulate those wars
This was a plot in one of the Culture novels by Iain Banks. Two civilizations decide to settle a dispute by fighting simulated wars in VR. Very interesting story.
> one of the Culture novels by Iain Banks

'Surface Detail' [0], by Iain M. Banks (the author uses 'Iain Banks' for non-SF work).

> Two civilizations decide to settle a dispute by fighting simulated wars in VR.

The war is between 'pro-hell' and 'anti-hell' supporters (hells are virtual environments where uploaded individuals can be subjected to eternal damnation). The war is a massive set of campaigns in various virtual environments with the loser agreeing to accept the overall result as a binding result. Things go wrong when one side begins to lose the virtual war and decides to escalate it into the Real.

[0] https://en.wikipedia.org/wiki/Surface_Detail

The Star Trek: TOS episode A Taste Of Armageddon is about this
And it fails when one side decides it doesn’t like the outcome and escalates to real war. We see it all the time at a very small scale, where the rules for resolution of conflict are overridden by not abiding by the rules. Unfortunately I believe we have to live with the threat of violence, and be prepared for it, as if we aren’t then those that are tend to win.
Right. War happens when you have a disagreement between two groups of people who care strongly enough about the issue that they're both willing to die in significant numbers to try to get their viewpoint to become reality.
That's whishfull thinking. Armies will just become more effective at killing innocent bystanders.
Why would my autonomous vehicles attack your autonomous army when I can just drop them in densely populated areas instead?
You wouldn't bother with autonomous drones if you wanted to mass-slaughter civilians. There are much easier ways, like chemical, nuclear, or incendiary weapons.
That's very cool. It also goes to show that if the US doesn't do it, then someone else will. I don't know how SV liberals think of these vehicles being used by Russia or China. Not trying to stir the pot or anything, just showing the possible consequences of the actions we've seen in the past months.
A true tragedy of the commons. I'm pessimistic that we would be capable of an international ban without rogue states and loopholes. We'll probably end up developing quasi-peacetime technology that could be co-opted for military purposes.
Yeah, this type of thing seems very easy to disguise as police or riot control equipment, and then just swap the gun from rubber bullets to real bullets if a war starts.
Estonia is a NATO country, just for the record.
I'm torn between the awesomeness and the morality of working on such a project.
I wonder what kind of military technology is acceptable in today's standards of morality. This is just a drone. It's not autonomous. It's a remote control car with weapons or radars mounted on top, controlled by a human looking through a camera. The military use case would be to reduce the possibility of human casualties.

Not sure how better to word this. I literally do not understand what kind of military technology is acceptable. Is the common position that no war should exist, no guns should exist, no conflict should exist, and anyone else anywhere in the world that wants to hurt other people should be allowed to without repercussion? Is the idea that if we don't invest in military technology countries like Russia, China, North Korea, and Saudi Arabia will just make batteries and toys? What is the position I'm supposed to be aligned with?

More accurately, it’s to reduce casualties for the military that uses them. It’s intended to increase casualties for the enemy.

Edit: it’s hard to reply when you keep editing your comments. It’s now quite different to your original post, though this comment does mostly make sense.

But that's what war is. I would imagine it's easier, and less of a risk to all humans involved on both sides, to create drones with non-lethal capabilities than to send non-lethal human forces into warzones.

Edit: I added the second paragraph to my original comment within 3 minutes of posting it, as I noticed there wasn't much hope of getting a response without fleshing out the question. I now have to choose between adding an additional reply to contend with the idea that I "keep editing my comments", or to edit this comment to deal with that additional claim.

I wonder though if the future will turn into the Star Trek episode where people are "killed" by an AI and then voluntarily submit to suicide. If war is too non-lethal it doesn't solve the reason for the war in the first place and eventually will lead to lethality anyway. As Sherman said "War is cruelty. There is no use trying to reform it. The crueler it is, the sooner it will be over." Perhaps more effort should go into eliminating the reason for war in the first place, which is of course easier said than done.
Hopefully it would be more like... eventually technology becomes so dominant on the battlefield that without it it’s just suicide. So as soon as one side’s drones are wiped out, they just surrender.

Although I suppose many countries would continue to resist in a Gandhi-like moral appeal to the victor’s population to stop the slaughter.

Pfff, that will never happen. Western people no longer understand the nature of war.

Even if one man with a nuke can effortlessly kill a million, a man with an axe can kill him as easily if he can manage to sneak close.

Regardless of the fact that conflict is inevitable, the idea that I designed a device to take human life would probably weigh on my concious.

That's not to say I haven't considered it.

Theres nothing immoral about weapons development.

Warfare and its plagues has an always been a human problem.

Unless it violates personal beliefs akin to any conscientious objector.

I say this on a technology forum because, while i am a technologist now, i worked a past career doing weapons tech with bright and moral people. Weapons tech development has an always been a scary realm bc any force/battlefield superiority almost ALWAYS results in its use...anf subsequent loss of life, combatant and otherwise.

Its a best of the worst kind of scenerios.

(Genuinely curious about how different people become confident in their moral and ethical convictions.)

Weapons development is not immoral because we have always waged war?

There's a great paradox with weapons. Imagine a world where somehow we agreed to stop development of any and all things that could be even somewhat effectively used as weapons, all the way down to the butter knife. The problem you've created now is that if we have just a single person decide to go against this and start developing weapons he would be able to forcefully impose his will as he saw fit with very little that could be done about it. And this is even worse when you take the analog away from individuals and consider it in terms of nations. Imagine one nation goes this direction. Now suddenly this one nation, which is probably a bit below par on the ethics meter, would suddenly be able to forcefully impose its will on the entirety of the world.

This means that it is a paradoxical imperative for ethical nations to maintain and develop as powerful of weapons as they can, even when they have absolutely 0 intent on ever using those weapons. Because otherwise, they risk falling victim to the lowest common denominator that does intend to use them. In reality this all gets really messed up, because as the person you were responding to said weapons almost invariably end up being utilized, often amorally, regardless of the player considered. So you're left in a scenario where if you don't develop weapons then you risk conceding the world to the least ethical player there is, yet if you do develop weapons then you are all but accepting that they will probably be used in an unethical fashion at some point.

Two pretty awful decisions, but I think one is worse than the other.

Since I am an American citizen, I am particularly interested in this question with respect to my country of citizenship.

Would you consider the United States an ethical nation?

Which nations do you consider to be ethical when it comes to this question?

Your argument makes sense in the abstract. However, I see the potential for inapplicability when applied to “real” nation states.

I'm not OP, but from my point of view, the United States has been one of the most likely nations in the world to use its weaponry on others (by my count, it's currently involved in 7 military operations [1]), and has been involved in dozens more over the last few decades. It has formulated dubious reasons, and in some cases outright fabrications, in order to justify some of these military actions, and in doing so, has destroyed countless lives, a number of those innocent ones. That's not ethical.

The reality of the world is that yes, there are actors out there who will do others harm if they can, and one of the duties of a nation is to protect its citizens. I am not opposed to a true defense force, to protect against invasion. But once that military power is projected outwards in order to fulfill political and economic goals unrelated to national defense, then a line has been crossed and that behaviour can hardly be considered ethical any more.

I should note that I don't consider my nation, Australia, to be ethical on these matters either based on past behaviours (sometimes being involved in these very same conflicts).

[1] Afghanistan, Iraq, Syria, Somalia, Libya, Niger, Yemen

You assert that there's nothing immoral about weapons development, and go on to almost directly contradict this- noting that "force/battlefield superiority" tends to lead to the use of these weapons, killing people. I really don't see what your point is; are you trying to argue that an occupation that leads to unnecessary human deaths is moral?
Unnecessary human death is a function of the policy makers. The weapons development is an enabler, but the 'why' behind the use of these weapons is where the moral obligation resides.
My prediction is the largest market for this will be Arab armies. Unit cohesion is the biggest problem they face and they usually abandon their equipment when faced with force. [1]

* [1]: https://www.youtube.com/watch?v=cZk4Yu42g0I

My Father was in Africa and in the middle east in the 50'ies and 60'ies. I remember him telling about how arabs would usually take off their boots and flee the scene in combat situations. I assumed he was joking, but maybe not...
Fleeing from combat I get, it’d probably be my instinct too. But why take your boots off? I personally run faster in shoes.
Would it be out of place to assume you haven't experimented much with barefoot running? What you say is intuitive, but in my personal experience incorrect. It took about 6 months for my body to adjust, but now running barefoot is far superior (or with minimalist shoes like the VFFs)
I wear VFFs daily, have for about half a decade at this point.

On uneven ground in a combat zone, littered with god knows what and where a broken ankle might mean death? Boots please.

> I wear VFFs daily, have for about half a decade at this point.

In that case, I would be out of place to assume you haven't experimented much with barefoot running :-)

> On uneven ground in a combat zone, littered with god knows what and where a broken ankle might mean death? Boots please.

That is a good point, tho I don't see boots making you faster , but I concede that the ankle support and steel toes would certainly be safer, and you're gonna be a lot slower with a broken ankle. So I still think barefoot you'd be faster, tho certainly less safe. If I were fleeing in battle, I might would rather gamble my safety for some speed, but it's probably highly situational. I suspect I've just wasted a lot of time analyzing a very unlikely scenario :-)

(comment deleted)
One thing that video doesn’t address is why ISIS is successful, while others aren’t.
I’m guessing unit cohesion from mutual belief in an extreme fundamentalist religion, lack of fear from the same, and horrible torturous death for not doing as commanded.
I would agree, tho I'm not an expert in this area but have studied it in the past.

It's difficult for non-radicalized people to understand the mindset of a radicalized religious person (not limited to Muslims). Their belief is truly powerful enough to overcome the innate human fears and drive for self-preservation that we all have. It's both admirable, amazing, and terrifying at the same time.

The threat of extreme tortuous death at the hand your own group is also a strong motivating factor that keeps people from defecting should their faith wane or fail them. They typically witness the horror of torture first-hand, and don't want any part of that themselves.

ISIS was/is not tribally-organized like the Arab regimes, and consisted mainly of volunteers, and many (most?) were foreigner.
ISIS is an Arab regime by most definitions.
That seems pretty clear - ISIS fighters were mainly volunteers, at the very least the vast majority had a deep, fundamentally motivating vision in the formation of a new Caliphate.

For Arab nations, on the contrary, there is very little in terms of a cohesive national identity worth fighting for, especially in places like Iraq, Libya and Syria. Thus, soldiers in their armies are basically just taking a paycheck (especially since the economy is shit otherwise). In the face of horrible torture and execution by crucifixion, not hard to see why soldiers would collectively believe it's just not worth it.

Although the equipment will still need to be maintained in the field, which means needing lots of technically trained people unless they want to just leave thousands of these littered around for the simplest breakdown.

Could be useful in magnifying the power though if a small number of well trained people, or a small ruling caste. Less vulnerable to losing power due to lack of military manpower.

Unless machines can fight autonomously at brigade or division level, it probably won't make a difference. Iraqi generals defending Mosul in 2014, ran away from attacking ISIS terrorists despite outnumbering them 15 to 1 [1]. All it would take is a panicking Arab general to press a button and the machines would retreat.

https://en.m.wikipedia.org/wiki/Fall_of_Mosul

When AIs wage war against each other in the real, what are humans supposed to do?
Try to avoid being ethnically cleansed by whoever owns the AIs and wants the Ukranian wheat fields or Baku oil fields.
Why, watch it on television, of course; take bets in the pub.

Imagine it. Two fully automated, autonomous armies, duking it out, no one getting hurt. Those with the best engineering and software win!

Winner takes all, but at that point, what is there to take? Resources? Land? The only thing left is the vanquished enemy's automated military tech and their engineers and programmers.

And the end-game? One peaceful world government under our no doubt Estonian overlords.

It will be Battlebots...for keeps!

I've read the book The Dictators Handbook (watch the CGP Grey video Rules for Rulers[1] for a good summary) and I can imagine a number of really clear and compelling Job Stories for this technology:

* As a ruler facing large-scale protests, I want to clear the crowds of people from the streets without deploying soldiers who might choose to actually side with the protestors.

* As a ruler who is afraid of a coup, I want to reduce the personnel-management costs of maintaining my military so that I can reduce the number of officers needed to maintain the same level of force lethality. This will let me purge my officer corps of possibly-disloyal members and lavish rewards on those who remain.

* As a ruler from an ethnic/religious minority, I want to restrict membership in the military to that minority without reducing its lethality so that I can prevent members of the majority from having trained members who could seize power or resist marginalization.

Technological advances like quinine and the Madsen and Maxim Gun allowed fewer than 1,000 Europeans[2] under Belgian King Leopold to spread the benefits[3] of civilization to The Congo, an area more than 3 times the size of Texas. I am excited to see the world that these new advances will usher in.

[1] https://www.youtube.com/watch?v=rStL7niR7gs

[2] https://en.wikipedia.org/wiki/Force_Publique#Under_the_Congo...

[3] https://en.wikipedia.org/wiki/Congo_Free_State#Mutilation (although yes, also https://en.wikipedia.org/wiki/Congo_Free_State#War_with_Arab...)

Wont the risk for officer rebellion increase with decreased size of the chore?

Even so, it's probably more likely that some murder bot engineers rebell against the ruler than thousands of soldiers and officers who have a harder time organizing than a smaller group.

Obviously, this apply for 'evil' rebells vs 'good' governments too.

I would not like to have a single point of failure murder bot army as garrisons.

As a bonus, developing-world tyrants usually have poor logistical and maintenance capabilities, so PMCs and defense contractors have the option of long-tail contracts for supply, maintenance, and upgrades as well, reducing the risk of regime elements going rogue with the murderbot army. It’s really a win-win for murderous ethnonationalist authoritarianism and unfettered global capitalism!
The US was researching comparable systems (and smaller) back in the mid 2000s. I knew a few people involved in those programs and seriously considered joining. Things went very quiet on that front, I've heard no news on those programs in years. I don't know if that means they continued behind closed doors or were put on indefinite pause.

This is the best article I could find on the subject which gives some background as to what derailed the program. http://strangehorizons.com/non-fiction/articles/what-killed-...

(comment deleted)