Sad truth: it WILL be secretly developed with or without ban... We just discovered yet another self-destruction method... In Smalltalk you could destroy everything with: "Smalltalk become: nil"
Application of unrefined AI in the context of deadly machinery is a real and immediate AI-danger. Self-refining, wolrd-dominating AI is at present time just a distraction from this.
Not really, the stakes are completely different. Robot armies might change warfare and balance of state power. They're not going to threaten humanity as a whole.
In a hundred million ways. Systems could be hacked, or humans social-engineered to start a nuclear war or manufacture and release a deadly plague, as some most trivial and devastating examples.
Also, threatening humanity is boring and irrelevant. What counts is threatening technological civilization. If we collapse to preindustrial times, it's game over for us anyway.
If we have a civilization collapse event, then the terminators are done as well. They will be dependent on our technology. The idea of an AI run world is just too far away.
I can imagine solar powered ai-run portable snipers covering large territories and murdering a lot of civilians over an extended period of time. Think forgotten land mines.
It can be worrying to think that aowrson could right now mount a gun to a drone and have it point at things it thinks look like people and shoot.
Then I remember that with any amount of technology they could just kill me themselves. Getting hit by a car is a much bigger risk, and the application of AI to prevent that is a much larger positive than the negatives AI introduces.
That must mean you are not in a middle eastern country. It's self centered to only have concern for your own well-being and myopic to think that, in a world full of self-centered people, with this tech being mass produced, that you'll continue living in a northern-atlantic utopia, where the probable causes of untimely death remain unchanged.
And it did occur to me, but I suppose I'm talking about things that we expect to change our way of life. I dare say if I was living in a third world country I'd have those problems now.
I hope and believe that the forces using AI for good will outweigh the forces using AI for evil.
As an American I'm going to go out on a limb and say a big part of this is: Americans in particular are absolutely insulated from mass death. Why wouldn't we be? We've never fought major wars on our own soil in over a century. We haven't had major disease outbreaks killing mass people in like 100 years. We cannot comprehend it; the idea that mass amounts of people die (1,000,000 iraqis, 100,000s of people in warzones like Syria) in relatively short time spans is impossible for us to comprehend.
This is the same reason why after 9/11, where a few thousand people died, people all over the country were literally having shared hysteria -- to the point of microwaving mail envelopes (Anthrax attacks!) or buying duct tape and plastic wrap to seal windows (Gas attacks!) They told kids in school about this shit and to prepare for it. It was hysteria, it was not isolated.
I think most people willingly choose to believe it doesn't happen at all. A few bombings here and tactical strikes there, twice (or ten) times a day, in a war for 15 years. "Not that bad".
Meanwhile we sit around and dream up AI scenarios where they're either A) Sci-Fi novel hellmachines who are bent on total domination of reality unless we "Fight the Machines" or B) useful, peaceful Roombas and car services that run us around and do all our chores and can never do any wrong. Look, my car can get my groceries for me! How fantastic is that! (provided your grocery store was not blown up by a Hellfire missile last week, of course)
The third possibility, C, that hostile, unrefined AI will be used to continue to enforce colonial and imperial desires around the world, killing huge amounts of people? Not possible. It's not fanciful or fantastic enough to want to believe, so people don't. It's not a Steven King novel or an Elon Musk jerk-off fantasy. It's just boring old poor people dying.
You only need to watch any discussion on Hacker News about "The threat of AI" to see how quickly people start talking about self-imposed God-machines, or talking about Ultra Roombas that do all your earthly chores -- while ignoring murder-robots in 3rd world countries, manned by massive armies. One of those is a fantasy dreamt up by 10 year olds, the other is a reality that already happens today with drone warfare.
You've forgotten the logistics chain. A "hostile" AI without any human support is going to run out of fuel, ammo or spares very very quickly.
The real threat from AI is automated colonialism. An empire where the people running it never have to get their hands dirty or face rebellion from troops suffering from post-atrocity PTSD. Send in the killbots followed by corpse disposal Roombas. No (human) witnesses.
It's hard to hide from IR using insulating clothes, because of the heat building up inside. But an insulating suit with active cooling might be an interesting idea to explore.
instead of WWII films with people hiding in mud for a tank to pass by so they could stick a bomb on it, we'll get WWIII films with people hiding in mud with an ethernet cable
Oh, you reminded me of that horrible horrible TV Series "Scorpion". Could not get past the second episode and watched the second only too see how much more stupid can it get.
But it couldn't be worse than the Independence Day movie from 1996. In that one, the aliens have huge ships; the mothership has one fourth the mass of the moon. Fortunately, Jeff Goldblum figures out how to upload a computer virus into the mothership. My memory of the plot is kind of sketchy, since I certainly wasn't tempted to see it again.
https://en.wikipedia.org/wiki/Independence_Day_(1996_film)#P...
I'm curious if the author would also refer to the M4 carbine as an "infamous killing machine," given the opportunity.
The fact of the matter is that the US is also developing these systems, albeit with more focus on autonomous air combat vehicles. To say that this is a Russia-only problem vastly understates how serious of an issue this is.
But it's not the same, the article makes it clear here:
"Known informally inside the corridors of the Pentagon as 'the Terminator conundrum,' the question being asked is whether stifling the development of these types of weapons would actually allow other less ethically minded countries to leap ahead?"
As a not American, I feel totally safe when it's the ethically minded USA who does it.
That was the question until the Kalashnikov announcement.
Now the question is, "given that everybody from Putin, to terrorists, Maduro, and North Korea have these weapons, do you want the US government facing them without such weapons ?"
In an arms race, you can perhaps refuse to pull ahead (even that is risky), but nothing good will come of falling behind.
If that's supposed to work like a threat to keep others in line, then of course, the US should work to (at least) maintain and (preferably) increase that power. Otherwise the threat won't matter. In other words, this depends on developing and demonstrating any and all weapons capabilities before others develop them, because if just one such technology provides an overwhelming advantage ...
Furthermore, there can also be no doubt that the US would use such weapons to maintain it's own and it's allies' security. There can be no doubt that we'd fire such weapons at Russian armies attempting to invade the Ukranian parliament for example. More controversially, there can be no doubt that we'd fire them at not really civilian Palestinians if Israel's army loses control.
The M4 isn't infamous though? I don't even know which one that is. The M-16 is famous but associated more with militaries than with guerillas or terrorists. I can't really think of a US-made weapon with a reputation for disreputable uses - maybe the "Tommy Gun" (associated with gangsters, at least in the popular imagination)?
You can see plenty of M-4/M-16/whatever (let's say AR-15 based weapons) in hands of bad guys in middle east, far east and africa. it's not like arms dealers have much morals to start with
The M4 and M16 are both versions of Eugene Stoner's AR-15 design. You know which one that is, even if you don't know you know it. Think about the last movie you saw that was made since 1980 and included American soldiers. What rifle were they carrying? There you go.
There also exist many AR-15-pattern rifles in the civilian market - obviously without the full-auto and burst-fire modes one finds in military models, but otherwise quite similar. They're generally satisfactory firearms - cheap to feed, not unpleasant to use or overly difficult to maintain.
They're also a longstanding bête noire of the anti-guns crowd, among whom the type is infamous because people have used them in a few mass shootings, and also because if you're afraid of firearms already then this type looks super extra scary - dressed in full tacticool ribs and Picatinny, it looks like something a Terminator might carry, and they're usually finished in black.
> You know which one that is, even if you don't know you know it. Think about the last movie you saw that was made since 1980 and included American soldiers. What rifle were they carrying? There you go.
See I thought that was the M-16; I could even have told you "the civilian version is the AR-15 and doesn't have full auto" but I'd never heard it called an M4 before. (Speaking as someone from a non-gun country whose only exposure to any of this is popular culture).
That's not too far off base - just that the AR-15 came first, followed by the M16 and later M4 variants on the military side, and any number of manufacturers' and homebrewers' variants on the civilian side.
Never heard of it, couldn't tell you anything about it - it's not one you see in movies / on flags / what-have-you (indeed the IRA are rarely seen in movies full stop - perhaps because they were allegedly Hollywood-funded?)
AR-15 base weapons (M16, M4) are much more accurate. So there's an advantage at distance. Also it uses smaller, faster rounds that have less stopping power, but are easier to carry. So there's a greater chance of running out of ammo with an AK-47.
From the pics it looks like some of these are HMG mounts I could see Tanks and EVFs having an Ai controlled secondary turrets to reduce crew size. The Soviets did like multi turreted tanks see the T35
Which would have to face the accompanying infantry as well Tanks don't do very well in FIBA or Fish And Chips (Fighting In Someone's House, and Causing Havoc In Public Spaces)
I'd say using autonomous machine gun on a light truck might be useful - no need for a human to risk his life, not to mention this thing will have better aim.
To put some context: we Russians always preferred automated systems to having humans in the loop — we have a lot of history with particularly prominent examples of less than fully sane humans in the loop, without proper checks and balances to restrain them.
So I am absolutely not surprised of the development, and we'll see much more instances of these systems soon. The only good part is that Russians strongly dislike complicated control systems, so the "AI" in question will be much simpler than what would Americans do.
Russians' dislike for complicated systems sounds very good but I hope it will have humans watching over it too. There were already many close calls and accidents (some fatal) due to computer and programming errors in both civilian and military systems.
No need. We have first-year privates with USB sticks instead. (And our USB sticks are, of course, plug-in incompatible with Western USB sticks, and weigh at least 15 kilograms, which greatly aids security).
Why would Kalashnikov choose to do a dog and pony show? Is it like the North Korean missle parades and a show of power for Russia? Do they just plain up sell these to certain countries and it's just like any other product launch? Does Kalashnikov have competitors that they are trying to look better than?
I'm not sure what I find more worrying, this, or the other end of the scale: the inevitability that it becomes technically easier and easier to turn cheap small autonomous drones into killing machines.
> Today, an opponent would try to fire back at the killing machine's "pilot," is that right?
If you're dealing with an armored vehicle, you're shooting at the vehicle. You can't shoot at the people operating it, because the armor is in the way. If you don't have anything to hand that can deal with the armor, you find something else to do instead. If you do have something that fits the bill, it's not going to leave the vehicle in an operable state after you use it.
Killing the crew of an IFV or tank during a combat engagement, and then stealing the vehicle, is something that happens in movies and video games - in real life, not so much.
I read that there have been cases of poorly briefed Ukrainian tank crews escaping and leaving behind tanks that had huge holes in armor despite the armor working as intended and preventing damage to anything important and tank still being fully operational.
Separatists took these over and patched/replaced the broken armor piece and used them.
It might all have been bullshit though and I can't find that article from years ago(and it wasn't in English anyway).
Closest I can find is the claim that an IS-3 from late World War II has been reactivated from a war memorial and possibly sent into combat - where it would be unlikely to survive the modern environment for very long, I hasten to note, despite being apparently pretty good for its day.
It's not implausible that a modern weapon could punch a hole in tank armor and nonetheless not only fail to achieve even a mission kill, but leave the crew alive. It is really, really improbable, though. I could buy that it'd happened once, although I'd like to see some kind of cite for that. I'd have a hard time believing it had happened twice in the same, as yet rather short, conflict.
I have found the article but it's in Polish. I also have no idea how reputable source this is, they seem very good on the first glance. There are some website writing about 'military' that are basically right wing trash or propaganda or conspiracies (or all of the above).
The hole was apparently not clean through because the original armor and extra layer and reactive armor installed during 'modernisations' did it job and protected the tank and the crew, it just now has this horribly looking hole in it so the Ukrainian crew abandoned it and separatists took it. This is a tank from the 60s, modernised in the 80s, by the way.
English Wikipedia also mentions 65 T-64s variants have been captured in Donbas (but the link is to a Russian website and I read Russian way too slowly and need a dictionary to do it so I can't confirm the citation):
https://en.wikipedia.org/wiki/Equipment_of_the_Ukrainian_Gro...
There are two significant parts (IMO) to the trend towards autonomous weapons. They're very different.
This article seems mostly concerned with autonomous/AI controlled decision making: picking targets and firing on them.
The 2nd part taking combatants out of the battlefield. This is more significant, IMO. This applies equally to autonomous & remote operated weapons. For thousands of years, fighting wars meant endangering soldiers. As this changes, all sorts of dynamics change, and conflicts may become more
Think of drones. Whatever the tactical implications, I think the strategic implications have been more important. The US is willing to deploy remote operated drones in scenarios where they would not have committed troops, even airforce only.
Look, you have to understand something: in Russia, human life has no intrinsic value at all, so it's not a conundrum in any way. New weapon can kill more people, all by itself? Great, bring more of those!
If you don't believe me, just read up on Russian history. Maybe start with Ukrainian genocide in 1932-33.
There is a certain inevitability about this. It will and should make many uneasy but like everything else autonomous weapons will advance.
Constraints have only worked so far for bio-chemical weapons and even there we don't know who is doing what. Nuclear weapons are easier to restrict because some sort of control can be effected on the underlying technology.
The only thing with both the above is the cost of using them is too high, only one country has used nuclear weapons against a population and it remains an unconscionable blot for many of us.
Autonomous weapons are much more accessible and will be integrated into modern conflict and law enforcement, shifting the balance in many scenarios.
> In the imminent future, the Group will unveil a range of products based on neural networks," said Sofiya Ivanova, the Group's Director for Communications. "A fully automated combat module featuring this technology is planned to be demonstrated at the Army-2017 forum,
So there's no actual product yet, and there's nothing known about the future one except that there are "neural networks" involved, whatever it means. Clearly, there's enough information to make wide-ranging speculations!
92 comments
[ 2.5 ms ] story [ 158 ms ] threadI agree with the GP I always said that a dumb AI with sufficient agency in the real world is much scarier now me than some super intelligence.
The former is much more likely to be realized and it already has to some extent and unlike the latter cannot be reasoned with.
Also, threatening humanity is boring and irrelevant. What counts is threatening technological civilization. If we collapse to preindustrial times, it's game over for us anyway.
Then I remember that with any amount of technology they could just kill me themselves. Getting hit by a car is a much bigger risk, and the application of AI to prevent that is a much larger positive than the negatives AI introduces.
And it did occur to me, but I suppose I'm talking about things that we expect to change our way of life. I dare say if I was living in a third world country I'd have those problems now.
I hope and believe that the forces using AI for good will outweigh the forces using AI for evil.
This is the same reason why after 9/11, where a few thousand people died, people all over the country were literally having shared hysteria -- to the point of microwaving mail envelopes (Anthrax attacks!) or buying duct tape and plastic wrap to seal windows (Gas attacks!) They told kids in school about this shit and to prepare for it. It was hysteria, it was not isolated.
I think most people willingly choose to believe it doesn't happen at all. A few bombings here and tactical strikes there, twice (or ten) times a day, in a war for 15 years. "Not that bad".
Meanwhile we sit around and dream up AI scenarios where they're either A) Sci-Fi novel hellmachines who are bent on total domination of reality unless we "Fight the Machines" or B) useful, peaceful Roombas and car services that run us around and do all our chores and can never do any wrong. Look, my car can get my groceries for me! How fantastic is that! (provided your grocery store was not blown up by a Hellfire missile last week, of course)
The third possibility, C, that hostile, unrefined AI will be used to continue to enforce colonial and imperial desires around the world, killing huge amounts of people? Not possible. It's not fanciful or fantastic enough to want to believe, so people don't. It's not a Steven King novel or an Elon Musk jerk-off fantasy. It's just boring old poor people dying.
You only need to watch any discussion on Hacker News about "The threat of AI" to see how quickly people start talking about self-imposed God-machines, or talking about Ultra Roombas that do all your earthly chores -- while ignoring murder-robots in 3rd world countries, manned by massive armies. One of those is a fantasy dreamt up by 10 year olds, the other is a reality that already happens today with drone warfare.
The real threat from AI is automated colonialism. An empire where the people running it never have to get their hands dirty or face rebellion from troops suffering from post-atrocity PTSD. Send in the killbots followed by corpse disposal Roombas. No (human) witnesses.
How about an AI that convinces the humanity to build it an army of androids and then wages war against its creators? Dibs on the idea!
In fact I should be writing a business plan for that business now.
https://blog.openai.com/adversarial-example-research/
[1] https://twitter.com/GrantTurnbull_/status/890513588787572737
https://www.youtube.com/watch?v=Hzlt7IbTp6M
But it couldn't be worse than the Independence Day movie from 1996. In that one, the aliens have huge ships; the mothership has one fourth the mass of the moon. Fortunately, Jeff Goldblum figures out how to upload a computer virus into the mothership. My memory of the plot is kind of sketchy, since I certainly wasn't tempted to see it again. https://en.wikipedia.org/wiki/Independence_Day_(1996_film)#P...
The fact of the matter is that the US is also developing these systems, albeit with more focus on autonomous air combat vehicles. To say that this is a Russia-only problem vastly understates how serious of an issue this is.
"Known informally inside the corridors of the Pentagon as 'the Terminator conundrum,' the question being asked is whether stifling the development of these types of weapons would actually allow other less ethically minded countries to leap ahead?"
As a not American, I feel totally safe when it's the ethically minded USA who does it.
Yep. That was sarcasm, sorry.
Now the question is, "given that everybody from Putin, to terrorists, Maduro, and North Korea have these weapons, do you want the US government facing them without such weapons ?"
In an arms race, you can perhaps refuse to pull ahead (even that is risky), but nothing good will come of falling behind.
Furthermore, there can also be no doubt that the US would use such weapons to maintain it's own and it's allies' security. There can be no doubt that we'd fire such weapons at Russian armies attempting to invade the Ukranian parliament for example. More controversially, there can be no doubt that we'd fire them at not really civilian Palestinians if Israel's army loses control.
I don't think an autonomous Kalashnikov can overwhelm a squadron of drones, which aren't the most powerful thing in the US arsenal either.
> There can be no doubt that we'd fire such weapons at Russian armies attempting to invade the Ukranian parliament for example.
Haha well, they'll have to keep invading and leave just the parliament alone then.
There also exist many AR-15-pattern rifles in the civilian market - obviously without the full-auto and burst-fire modes one finds in military models, but otherwise quite similar. They're generally satisfactory firearms - cheap to feed, not unpleasant to use or overly difficult to maintain.
They're also a longstanding bête noire of the anti-guns crowd, among whom the type is infamous because people have used them in a few mass shootings, and also because if you're afraid of firearms already then this type looks super extra scary - dressed in full tacticool ribs and Picatinny, it looks like something a Terminator might carry, and they're usually finished in black.
See I thought that was the M-16; I could even have told you "the civilian version is the AR-15 and doesn't have full auto" but I'd never heard it called an M4 before. (Speaking as someone from a non-gun country whose only exposure to any of this is popular culture).
There's a category for cheap, small-caliber pistols, "Saturday night special".
That gives it a worse reputation than most other rifles.
That doesn't sound quite as bad or threatening - although probably very bad news for anyone trying to take on such a vehicle at close range!
So - tank support (instead of sending infantry), clearing missions when you don't really care - just shoot everything that moves, defence turrets like this one - https://www.techcult.ru/content/2016/3690/robot-chasovoj-FLf....
I'd say using autonomous machine gun on a light truck might be useful - no need for a human to risk his life, not to mention this thing will have better aim.
So I am absolutely not surprised of the development, and we'll see much more instances of these systems soon. The only good part is that Russians strongly dislike complicated control systems, so the "AI" in question will be much simpler than what would Americans do.
https://en.wikipedia.org/wiki/Stanislav_Petrov
Case in point: https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar... if it wasn't for Stanislav Petrov the world might have ended back then.
These comments though.
With these weapons, the only defense is shooting at the machine itself, breaking it.
That reduces your chance of stealing the killing machine after taking out its pilot.
If you're dealing with an armored vehicle, you're shooting at the vehicle. You can't shoot at the people operating it, because the armor is in the way. If you don't have anything to hand that can deal with the armor, you find something else to do instead. If you do have something that fits the bill, it's not going to leave the vehicle in an operable state after you use it.
Killing the crew of an IFV or tank during a combat engagement, and then stealing the vehicle, is something that happens in movies and video games - in real life, not so much.
Separatists took these over and patched/replaced the broken armor piece and used them.
It might all have been bullshit though and I can't find that article from years ago(and it wasn't in English anyway).
It's not implausible that a modern weapon could punch a hole in tank armor and nonetheless not only fail to achieve even a mission kill, but leave the crew alive. It is really, really improbable, though. I could buy that it'd happened once, although I'd like to see some kind of cite for that. I'd have a hard time believing it had happened twice in the same, as yet rather short, conflict.
The hole was apparently not clean through because the original armor and extra layer and reactive armor installed during 'modernisations' did it job and protected the tank and the crew, it just now has this horribly looking hole in it so the Ukrainian crew abandoned it and separatists took it. This is a tank from the 60s, modernised in the 80s, by the way.
http://dziennikzbrojny.pl/artykuly/art,3,8,8725,pole-bitwy,1...
English Wikipedia also mentions 65 T-64s variants have been captured in Donbas (but the link is to a Russian website and I read Russian way too slowly and need a dictionary to do it so I can't confirm the citation): https://en.wikipedia.org/wiki/Equipment_of_the_Ukrainian_Gro...
This article seems mostly concerned with autonomous/AI controlled decision making: picking targets and firing on them.
The 2nd part taking combatants out of the battlefield. This is more significant, IMO. This applies equally to autonomous & remote operated weapons. For thousands of years, fighting wars meant endangering soldiers. As this changes, all sorts of dynamics change, and conflicts may become more
Think of drones. Whatever the tactical implications, I think the strategic implications have been more important. The US is willing to deploy remote operated drones in scenarios where they would not have committed troops, even airforce only.
If you don't believe me, just read up on Russian history. Maybe start with Ukrainian genocide in 1932-33.
Constraints have only worked so far for bio-chemical weapons and even there we don't know who is doing what. Nuclear weapons are easier to restrict because some sort of control can be effected on the underlying technology.
The only thing with both the above is the cost of using them is too high, only one country has used nuclear weapons against a population and it remains an unconscionable blot for many of us.
Autonomous weapons are much more accessible and will be integrated into modern conflict and law enforcement, shifting the balance in many scenarios.
Why even have a military. Just keep a few special forces and augment the rest with Autonomous/AI gadgets such as these.
Sell your defense shares.
> In the imminent future, the Group will unveil a range of products based on neural networks," said Sofiya Ivanova, the Group's Director for Communications. "A fully automated combat module featuring this technology is planned to be demonstrated at the Army-2017 forum,
So there's no actual product yet, and there's nothing known about the future one except that there are "neural networks" involved, whatever it means. Clearly, there's enough information to make wide-ranging speculations!