Landmines are simple and can't be hacked - a drone on the other hand is just software - Iranians spoofed the GPS systems of an US drone and made it land
The problem with mines is the indiscriminate murder. The mines can't be hacked, but they don't know the difference between an enemy soldier in $YEAR and a playing child in $YEAR+10
I think we can probably all agree that while these things are different, we're really just talking about different flavors of disastrous. Sure, right now landmines have the lead in unintentional injuries/deaths, but in the grand scheme of things it's the early days, and bad quality automation (or good quality automation put to awful tasks) has yet to show its true potential. Then again it's hard to compete with the 60,000,000+ unexploded landmines estimated to be out in the wild.
May we not see the day with 69M autonomous drones, recharging off utility lines or solar, and hunting humans relentlessly with shitty visual recognition algorithms.
I think 60M is the lower end of the estimate, which at the high end is around 75M, so I suppose 69 isn't a bad middle ground.
And yeah, as I wrote my comment I also envisioned self-recharging drones as a logical next step. If you're going fully autonomous, why keep any requirement to come back to base? The one in this story was kamikaze, but if you needed something with more of a punch that became expensive enough to require reusability, then you could have it come home, or just station autonomous ammunition stations at strategic locations.
Landmines are made to not be detected and last 100 years.
These fall to the ground after 30 mins.
Unexploded bombs also bury themselves.
These will be attractive to children (boys) but since they can only go 5 km the children are already part of the battle.
Sure I'd add a further mode to land early and have a low power sensor to blow up when it detects a person in some cases but there's little reason to think this story is even true yet.
Wow, adding some gender to it is beautiful. And how child labor is limited. I like that you went with a third paragraph of ideas instead of just self reflection and deletion. Strong take.
Unexploded ordinances predominantly kill young boys.
Even though it saves lives how we direct education campaigns, Twitter won't allow us to discuss this? What a world we live in, it's honestly falling apart.
For labour it's a farming and digging issue for adults but for the children killed it's mostly about play I'm not sure where this is coming from?
As I said given Kargu-2 are going to be on the surface or returned to base they will be quickly cleaned up. Also unless they have gone to suicide mode and missed they should be relatively harmless.
Without added programming they are not like mines.
Add a small radio receiver that turns the drone on when it picks up a signal from a handheld radio or cellphone within flight range. Add small solar panels to trickle charge the battery while it waits. What’s the shelflife of LiPo batteries?
> The drone, which can be directed to detonate on impact, was operating in a "highly effective" autonomous mode that required no human controller, the New York Post said.
> "The lethal autonomous weapons systems were programmed to attack targets without requiring data connectivity between the operator and the munition: in effect, a true 'fire, forget and find' capability," the report from the UN Security Council's Panel of Experts on Libya said.
I’m not sure why this is called “rogue” — apparently it was operating as intended
Probably "rogue" in the sense that direct control was not maintained at all times. The human controller told the drone to pull the trigger when the time was right, and the drone itself made the call as to when that was. It is a bit of sensationalizing for the sake of the headline though.
the problem is that "rogue" implies some kind of malfunction -- it acted against a stand-down order and killed anyways.
In this case, it seems the drone was supposed to do exactly that, and that's perhaps far more dangerous -- this is the first kill where the drone was approved to determine kill targets, and execute that decision, with no human to verify. This really shouldn't be a goal we have in the first place -- to remove humans from the decision loop -- and yet apparently here it is.
This is nothing new. The US military developed a naval mine that could launch a torpedo to kill targets without humans in the decision loop back in 1979.
Hm. That’s interesting — it’s an upgraded mine. I think the indiscriminate nature of it makes it more acceptable, at least to me; you’re creating a no-go zone for all — allies, enemies and nuetrals.
With this you’re creating a no-go zone for your enemies, and very likely safe for your allies (who you can tag properly with markers), but the difference between enemy and neutral is rather unclear (especially in the Middle East, where the line between combatant and civilian is very thin, almost indistinguishable even to humans on-site)
I believe the goal is driven by the desire for the drone to continue to operate even if humans are not reachable because of communications jamming - if you're fighting against some low-resourced guerillas, you can have humans in the decision loop, but if the loitering munitions/drones are to be used against an opponent with some air defence and EW/ECM systems (e.g. the recent Azerbaijan-Armenia war, where Turkish drones/loitering munitions systems were widely used, especially against AD systems) then an autonomous operation capability seems a quite desirable goal for practical reasons.
Guided missiles have to be told by a human what to strike. The dangerous thing that's happening here is that we trust the AI to kill someone all on its own without the operator having to identify the target.
Not really. There have been anti-ship and anti-armor guided missiles available for years which fly to a designated point and then independently search for a target to attack without a human operator. This is nothing new.
Well, now we have a ready-made army for the AI uprising: connected, autonomous, and relatively easy to mass-produce. Once The Final Program comes online, all it has to do is hack the right servers and it's got its airforce. Nukes are air-gapped, something tells me these are not.
Kind of obvious that you can't effectively ban this.
The best realistic option here, in my opinion, is to require sensible constraints. E.g. hard-wiring war zones in them - a separate circuit could prevent firing if the drone is outside certain area. You could decrease the attack surface for such system by relying on a Kalman filter getting it's data from a dedicated accelerometer and a compass.
You probably can't ban it in the sense of complete & total non-use. But collectively the world has mostly decided and actually followed through on not using certain things like chemical weapons in warfare. While this has not eliminated their use, it would probably be much worse without this prohibition. For example they went largely unused in warfare in the European theatre of war in WWII.
Of course most recently Syria is a horrible exception to this track record, and it remains to be seen if this will embolden others to do the same. Nonetheless, the prohibition does show that such agreements can at least reduce the usage of certain methods of waging war.
But I don't think hardware or software constraints be effective: Once one side of a conflict decides to break the a prohibition about using autonomous drones, I don't think they will care about hardware protocols. There's also the fact that warfare generally does not have static boundaries and no one willing to use this type of weapon would concede to their uselessness once the fighting moved elsewhere. Especially because their enemies would simply & quickly change their tactics to work outside that zone.
Right. You can't ban the technology, but you can out the nations who employ merciless weapons as uncivilized pariahs who deserve coordinated disrespect, isolation, and punitive measures until the practice ends.
I don't disagree with your point, but this part highlights how hard this is in practice. When does "the practice end"? When the war is one? When the current offensive ends? When we believe the devices aren't being produced anymore? This immediately reminds me of how Chamberlain treated Hitler.
You can sanction the countries in a variety of ways until they stop. You could deploy counter measures to prevent some things: a targeted strike against a chemical depot, or deploy anti-drone technology. This is not unlike the Student approach to slowing Iranian nuclear weapons development.
You can also go after the people behind the orders to do these things and bring them up on war crimes. I find the former option much more ethically reliable, as the later frequently suffers from the fact that war crimes tribunals are generally not convened against the actions of whoever wins, while the winner is all to eager to punish their enemies even beyond defeat.
But none of these options are perfect. We live in a world where bad people can do awful things and never suffer the consequences. Justice isn't dead, but it has never been equally applied.
Yes, though I think equally, and maybe the more effective aspect of the prohibition against chemical warfare was the general recognition that going down that pathway would be much more awful for everyone than sticking to conventional methods. This is partly why Germany pretty much avoided using them in warfare during WWII: Once the genie was out of the bottle they would lose any advantage obtained in their initial use. (I don't know if it's apocryphal, but apparently Hitler also disliked them from his own experience in WWI)
I can hardly be faulted for not having enough information to say if the prohibition will work on a longer time scale, and I even acknowledge this when I point out the risk that Syria's use may embolden others. I also fully admitted it did not have a perfect track record.
So you are faulting my argument for issues I already acknowledged and addressed in the context of my claim: for a weapon that is cheap and easy to produce we have seen fairly little use of it.
Regardless of what comes after this, we can say that the prohibition worked reasonably well for at least 100 years. I'd certainly be happy if we got even 25 more years before fully autonomous lethal drones became common place, and a prohibition on such weapons could buy us so e time on the issue.
> the world has mostly decided and actually followed through on not using certain things like chemical weapons
Chemical weapons are not used because they were ineffective even during WW1 and make even less sense now, when there is no trench warfare and targets are generally highly mobile. Conventions have little to do with this.
They are very effective in certain tactical situations. Area denial as an example. Or to slow an enemy. Or induce fear and morale loss. Or inflict countless non-lethal casualties that sap resources from the enemy: I have relatives who 100 years ago sat for weeks in hospitals trying to recover from mustard gas (they never did, not fully). Or to force the enemy to spend more resources negating them than you do to create then: The simplest are cheapest are easy to produce while effective gas masks for ever soldier are a little more expensive and not fully effective under active conditions of war. Use against civilian populations as a weapon of terror, as in Syria, is yet another one. Asymmetric warfare in general is a broad area of usefulness.
Certainly they were effective enough that Japan used them in thousands of attacks against Chinese forces. Effective enough that the US and Russia and probably others actively researched and stockpiled them for decades into the cold war. They were considered a significant threat by the US during the Gulf War: Iraq had a history of using them against Iran so the US took many precautionary steps to protect or inoculate their
soldiers against them. Too many precautions as it turns out because those protection methods are themselves credited as a major factor in Gulf War Syndrome which impacts hundreds of thousands of US soldiers. The US certain believed in their efficacy 70 years after the trench warfare of WW One.
I don't know where you get the idea that they are not effective, or only effective under conditions if trench warfare. Actual use shows this is not the case.
One could further raise an association with Nick Bostrom's vulnerable world hypothesis:
> One way of looking at human creativity is as a process of pulling balls out of a giant urn. The balls represent possible ideas, discoveries, technological inventions. Over the course of history, we have extracted a great many balls – mostly white (beneficial) but also various shades of gray (moderately harmful ones and mixed blessings). [..] What we haven’t extracted, so far, is a black ball: a technology that invariably or by default destroys the civilization that invents it. The reason is not that we have been particularly careful or wise in our technology policy. We have just been lucky.
This is a useful thought experiment but it ignores the fact that the balls are more like a tangled mass of burrs, all variously interwoven
Try to pull one out and you get a web of affordances and side effects and unforeseen interactions stuck to it
If we were to be wiped out by our technology I see no guarantee some extradimensional coroner would be able to point definitively to just one
Say it's climate heating, rapid-onset Venus mode. Was it the internal combustion engine? The coal plant? The automobile? The wheel? Language? Money? Industrialism? The limited liability corporation? How do you pick one?
No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms
Some balls are more webby than others. Your counterpoint is valid but it is additive: the discrete balls exist too in cases where there are significant stepwise breakthroughs.
Fair point, yes, the urn's contents are mereologically weird, neither wholly discrete nor continuous, liable to coalesce into orbs of manual or digital or social or intellectual utility which evolve in our minds and hands
Also some balls (e.g. nuclear weapons, stray asteroids) make it a lot easier to point to something immediate and mostly unambiguous
> No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms
I agree the analogy isn't perfect, but one can say that despite technologies not having a discreet origin, we can reason about them in retrospect. So maybe modern artificial neural networks were initially made possible by improvements in microscopy and experiments in cell culture to deduce the mechanisms of biological neurons, but we can reason about ANNs as a technology that is changing what is possible by a machine.
I imagine Nick would also be inclined to be fatalistic about technological progress, in that these "black balls" will inevitably be found. His main stance is that we should be enacting the global ethical frameworks around these technologies much quicker.
> despite technologies not having a discreet origin, we can reason about them in retrospect
Yes, agreed, good point
I also agree about Nick's probable fatalism, and I think his work is brilliant and valuable, however one thing irks me. He's made a successful career out of hypothetical "black ball" speculation while saying little about the potential for blackness of the tech we already have. He frames existential risk as if we are on the cusp of dramatic danger but still relatively safe. In truth we are already over the cliff & falling rapidly
Take his paperclip maximizer. They already exist! We need not wait for some future AI to realize a badly misaligned objective function. We're already determined to transmute the remains of our homeworld's long dead into plastic and atmospheric carbon
>there was some moderately compelling theorizing that they would set the entire planet's atmosphere on fire.
A very popular story but it was also primarily 1 of the 8 scientists working on the Manhattan project and only because he hadn't had all the math showing how incredibly unlikely it would be shared with him. It was already a settled "No, not going to happen." before the first test was conducted. The story ends up being a popular soundbite but with no real teeth on it. [0]
I think the idea is, if the math had worked out the opposite way, then nuclear tech would be a black ball. If there was a bomb we could make and if it went off it would ignite the atmosphere and kill everyone - that would be a black ball.
And still may yet be. The nukes haven't gone anywhere or become less world-ending. We are just having a moment where there's no serious risk of nuclear powers declaring all out war. But I don't recall inventing a new kind of diplomacy or geopolitics that makes war unthinkable.
One should also mention the anthropic principle in connection with this: the reason we haven't drawn a black ball is because we're here to observe that we still exist. Also of interest is the Fermi paradox: if (relatively high) estimates for the prevalence of extraterrestrial intelligence are correct, why haven't we found anyone? Taken together, it seems very likely that there is in fact a black ball in that urn, and we just happen to be one of the extraordinarily lucky planets that has not yet found it.
Oddly, this is kinda freeing in a nihilistic way. If you know that you and your species are going to die eventually (which you do, the eventual fate of the universe is heat death), you can focus on living for the present instead of brooding on the inevitable death of mankind, and just be grateful that you and your species happen to be alive at this particular time on this particular planet where life is actually a thing.
"THEY sent A SLAMHOUND on Turner’s trail in New Delhi, slotted it to his
pheromones and the color of his hair. It caught up with him on a street called Chandni
Chauk and came scrambling for his rented BMW through a forest of bare brown legs
and pedicab tires. Its core was a kilogram of recrystallized hexogene and flaked TNT."
In reality though, most of the munitions such as these will not be used for wars.
Unfortunately, I'd have to put my money on munitions like these being used more often for targeted assassinations of <insert name of prominent person here>. It seems on the surface, very effective. And I could see where it would be hard, if not impossible, to trace. I think people in the future will think twice before running for mayor or agreeing to be a sheriff or police chief for instance.
For example, once the cartels build some of these, I'm not sure what kind of odds I would give anti-cartel leaders. Probably would be even worse in the US given our acrimonious dichotomy. I could see tit for tat assassinations going on all the time.
Of course, all that also means that a lot of work will likely be put into countermeasures. So who knows how all this will develop I guess?
Wars happen anyway, someone might start a war with you and then you need to manage to defend yourself without killling too many civilians, if AI helps then why not?
an AI drone targeting and killing a person is a view through War on Terror lenses, a first world view so to speak. The Turkey and similar drones have been developed in a different context, in the context of real wars and military conflicts. The TB2 (operator guided) drones were key for Turkey succeeding in Syria, Libya and Azerbaijan - in all these cases they obliterated opposing, Russian backed, forces which were using Russian made air defenses (mostly Pantsir-S1) covering the Russian made armor. Ukraine has already bought these drones and started licensed production of them (including for export), and there are talks about further collaborative development of the drones with Turkey.
Now, what can oppose those drones, and in particular by the probable opponents in the region? Manned air fighters, other drones, and jamming. In all those cases to preserve and maintain superiority the drones have to get fully or at least partially autonomous. Note that the Turkey isn't participating in the stealth race even though they could have gotten F-35 if they behaved, and the countries like Azerbaijan and Ukraine for example have no chances of getting stealth, so drones is the [only] way here, and autonomous drones is just the next step in the development of those weapons what their national security (or regional superpower status in case of Turkey) will be relying upon.
And efficient fast coordination inside directly networked swarms may be well beyond human operators abilities. Especially in a swarm against swarm situation.
Manned fighters aren't really effective against small, low altitude drones and other loitering munitions. They're too small and slow for fast movers to engage. The US military is putting a renewed emphasis on short-range air defense (SHORAD) ground systems including jammers, missiles, guns, and lasers.
Someone already posted the slaughterbots video, but I just find it unsettling that the production qualities of this video, the intense music, the graphics, etc. remind me exactly of the corporate presentation in slaughterbots.
I'd almost prefer that this showed something like someone's head getting blown off their body, to demonstrate the actual brutality of what it is designed to do, over making it seem like some video game.
The entire defense industry avoids just saying what they mean. They'll work on a "computer recognition system for analyzing target response matrix" instead of saying "we want to know all the ways that someone will try to frantically get away from us in their last moments of desparate survival before we kill them, and kill them anyway"
They're a really twisted group of people that mostly feel like they have no other opportunities (that maintain their lifestyle). Not so different than people employed in any other industry, just important to understand.
ugh, that upbeat music choice for the presentation video made me slightly sick.
Had a similar reaction once before, after randomly walking in a military exhibition. While I understand the need for defence capabilities, and I can be impressed by the engineering, I don't get the triumphant glitter surrounding the showcase of killing machines.
What I find more disturbing is that there are people who willingly work on this and who get rich off of it.
You know, the other day I saw a news headline about one of the world's biggest arms dealers being arrested... and the first thought through my mind was, "What? They arrested the United States?"
Being a small fry, independent arms deal is grounds for imprisonment, but doing it as part of a nation-wide effort will get you wealth, medals, social status and government funding.
So I have two options. I can either hide the fact that I find that video incredibly cool, or just be open and honest about it.
If you’re wondering why it’s an effective video, it’s because it makes me want one. I don’t even have anything to blow up. That’s the essence of effective marketing.
We can also recognize that in the wrong hands, it’s an evil device. But so is a gun. And I would imagine the reactions to the first portable guns may have been similar.
Lots of people find guns cool, and I have lots of fond memories of shooting with my dad and a friend. Learned about gun safety and how to properly care for one. I also learned I wasn’t responsible enough to keep one, so I’ve never bought one.
This is obviously different. But it’s not inconceivable that in the future, you might be able to deploy a sentry quadcopter designed to chase down (not destroy) someone who just attacked you. I was shocked that there was a drive-by shooting in my neighborhood a few months ago. Would’ve totally sic’d a drone on them the moment I heard the gunshots.
Dunno. Sometimes I feel lonely for feeling this way, when the only acceptable social response is outrage or indignation. It’s cool to blow things up, and humans have had a long and proud tradition of making things go boom.
The key is to make sure that this tech won’t have unforeseen consequences. But as far as I can tell, the consequences appear to be foreseen in this case. It’s a drone that goes boom when next to someone.
All that said, I agree that it’s a terrible idea to let it decide to do this automatically. But in a warzone, it’s not impossible to imagine some heuristics that might make sense in certain highly constrained circumstances.
Of course, most dictators don’t really care about minimizing civilian casualties, and dictatorships seem like the target audience. But, ditto for guns.
> But as far as I can tell, the consequences appear to be foreseen in this case. It’s a drone that goes boom when next to someone.
The immediate second order effect is, people start buying drones programmed to destroy a drone seeking them out. This should be much less controversial than the first drone, since it's not operating on a human target. Now the first drone needs to be able to eliminate one -- and shortly, multiple -- drone(s) before then destroying the human target. The defending drone(s) will certainly increase in number and firepower. It's basically a straight line to automated warfare in the streets with no regard for collateral damage.
The hurdles, of course, would be regulation on the ability to purchase drones, and the permitted level of firepower. I don't like the idea of us negotiating the laws around that while drones are causing collateral damage in an urban (or any) environment.
Ground based weapons will probably be more effective drone defenses than other drones. It just isn't practical to keep defensive drones continuously airborne, and small drones lack the payload capacity that would be needed to reliably detect and intercept other drones.
There’s at least a third option, but hemlock is hard to find on the ground. This is definitely r/shithackernewssays material. Why is the Jeff Goldblum character from Jurassic Park so often quoted but so rarely heard.
This is literally about killing another human being and you went with “Well yeah but”.
I wonder how courts will ultimately classify AI face recognition when it becomes attached to lethal devices.
Tripwires attached to a claymore fall under boobytrapping laws under arguments of reckless endangerment. I've heard it reasoned as there's no mind behind the assault and so it's reckless even if it harms someone breaking and entering.
Facial recognition gets closer to an intelligence behind the weapon, but a less than human intelligence (less than canine?). At what point does your "tripwire" have enough intelligence to be legitimate under castle doctrine?
Man, I have such an opposite reaction. I know I could develop something like this if I tried but the thought of building something that would be used to kill someone, even a “bad” someone honestly makes me ill.
I don’t know. It didn’t look cool to me at all. I’d classify it as fairly boring.
It also kind of has to explode itself to do it’s job, which feels like a waste to me. So many things you can do with a drone that are immediately useless if it blows itself up.
> We can also recognize that in the wrong hands, it’s an evil device. But so is a gun.
I'm not sure what your point is here? The 'but' implies that we all now agree that guns are basically OK which is not universally true, and I would say increasingly the opposite.
Also your conception of 'the wrong hands' and other peoples' are not necessarily the same - one man's terrorist is another man's freedom fighter, and so on.
> But it’s not inconceivable that in the future, you might be able to deploy a sentry quadcopter designed to chase down (not destroy) someone who just attacked you. I was shocked that there was a drive-by shooting in my neighborhood a few months ago. Would’ve totally sic’d a drone on them the moment I heard the gunshots.
I think this is exactly what scares people - there are people who would sic killer drones on people Judge Dredd style, out of a misguided sense of moral duty; I heard about a recent case of someone using the 'stand your ground' laws to justify (and be celebrated for!) going to a neighbour's house during a robbery and shooting the thieves in the back, killing them both. Not to mention that in a future where these things are available, that drive-by would've instead been a fly-by killing and you might never know who did it.
It was scary as hell to have a drive-by happen. The cops couldn't do anything. They were long gone by the time the police came. And then they came back and shot up the same place again a few hours later.
If I had a drone that could track them, they may have been brought to justice. I don't see this tech the same way you do. I see it as a stepping stone towards a world where people can protect themselves and their loved ones, as they do with guns.
I think we'll just need to agree to disagree. I have a lot of things I could say, but I just don't get how people can calmly live in a country that was earned through bloodshed, while decrying those who were willing to lay down their lives for you.
This is a military device. It's designed to kill people, because this is one of life's unfortunate necessities. I think the people here who are saying that there's nothing honorable about killing someone should teleport themselves back to Midway in 1942 and tell your great-grandparents that, face to face.
> It was scary as hell to have a drive-by happen. The cops couldn't do anything. They were long gone by the time the police came. And then they came back and shot up the same place again a few hours later.
I don't personally care so much about the lives of people who engage in this kind of behaviour, but I do care about living in a peaceful society where violence is regulated by the law rather than being handed out vigilante-style. Arming everybody does not prevent gun violence, it enables it. You need only look at the blatant correlation between the number of guns per capita in the US and the number of shootings in the US. I'm in the UK - you know what we don't have lots of? Guns. You know what we also don't have lots of? Gun violence. It still happens, of course, but it's far rarer, and I don't feel the need to arm myself just to stay safe. That drive-by you were unfortunately so close to was made easier by increased access to weapons.
> If I had a drone that could track them, they may have been brought to justice. I don't see this tech the same way you do. I see it as a stepping stone towards a world where people can protect themselves and their loved ones, as they do with guns.
The trouble is, the easier it is for you to get hold of this kind of drone tech, the easier it is for everyone, which obviously includes those who would do you harm. See my comment above. You want a drone that can follow people around? So do a host of spurned lovers and over-protective parents and so on and so on - it's not the clear win for society that you seem to think it is.
> ...I just don't get how people can calmly live in a country that was earned through bloodshed, while decrying those who were willing to lay down their lives for you.
You seem to think I'm ragging on the military, but I'm not - perhaps you're referring to other comments? In general, one's neighbours shouldn't be laying down their lives for each other on a day to day basis unless they're in the military or the police, who are, at least in theory, trained and regulated in their use of force and when to deploy it (yes there are obvious potential problems there too, but that's a different discussion).
I would also like to point out that the bloodshed is supposed to be temporary - once the wars are won and safety is assured (I'll assume you're referring to the World Wars here), we're supposed to stop shooting each other.
> This is a military device.
So are AR-15s - how many of those are in the hands of twitchy civilian militia groups and other randomers with a grudge?
> I think the people here who are saying that there's nothing honorable about killing someone should teleport themselves back to Midway in 1942 and tell your great-grandparents that, face to face.
I'm willing to bet that the majority of those who served in World War II didn't see anything honourable about it either, just that it was a grim necessity, which it absolutely was. Honour and glory are concepts the armed forces use to encourage people to do necessary but horrible things - we should of course be eternally grateful for and respectful of their sacrifices, but I think it's actually disrespectful to the fallen to glorify war, as they gave everything so their descendents wouldn't need to engage in it.
After thinking a lot about your comment, there seems to be some contradictions in it.
In this case, you feel that war is a grim necessity, and also admit that this is a military device. If your country is in a war, wouldn’t you want to be on the winning side? Culture often doesn’t survive losing a war.
If so, then I don’t understand why people are against development of this tech. If you’re on the battlefield, you bet that you’d want a way to take out the people shooting at you from the hill on the other side. Usually that means calling in air support. Here, your air support can be almost pocket sized.
It’s pretty much as simple as that, and I think the crux of the disagreements. People seem worried that this tech will get out of hand. I am too. But it also feels inevitable.
Would you want to fight a war without guns? Because it sounds similar to an argument against developing guns.
Because you are only looking from your perspective.
What happens when people who did the driveby get the technology (if you can buy the drones so can the bad guys).
What happens when your autonomous drone decides to shoots neigburs kid for playing hid and seek, in a fashion that drone algorithms find objectionable.
And suppose your drone goes after them, finds them, then what ? You do realize that after, the drive by is done and they "flee" the scene, you can't just go after them and shoot them as a civilian? You can defend your property (someplaces ) but that does not include chasing after them.
Is the drone smart enough not to shoot when there are other people behind it's targets ? what if it's Halloween ?
Supose those drones get sold, and given current IOT security standards, someone hacks them ?
That is not valid. Lots of people like guns because they represent self-defense and protection of their property. Much fewer people like or own hand grenades or bombs, because they have a different purpose. This falls under the latter category. Just because they are all exploding device doesn't put them all in one bag.
The more humans are detached from defense capability, the worse it will get for non-authoritarian countries, because drone armies need only money and not legitimacy of the state
I'm guessing that you have never been on the receiving end of an explosive device.
This scares the shit out of me. Call me naive but the idea that somebody on the other side of the planet can set me as a target in a device that they just set and release is completely inhuman. If you want me dead, then do me the curtesy of at least putting your own arse on the line to achieve it.
The potential for this type of device to be cobbled together is well within the capability of pretty much any reasonably funded electronics hobbyist, and I think one of the reasons that we haven't seen it more frequently is that hitherto it's not been morally acceptable. If this type of weapon becomes mainstream in government arsenals then you can kiss that goodbye.
I'm not aware of any other story-- about US hardware or otherwise-- that has shown a drone operating solely upon its own programming to attack a person. Otherwise, yes, it's common knowledge that the US uses human operated drones for lethal missions, but there is a human and not programming behind the orders to attack.
Anti-Ship cruise missiles, for one. The TSAM (and possibly also the Harpoon) navigates to a search area and then determines a ship to attack within that area.
The first self-guided antiship missiles might also qualify, but they were (iirc) lock-on before launch weapons. (Operational in the Pacific theater during WW2, even!)
How many years until average person could make one of these? How do you protect politicians, reporters, famous people? I suppose we can assume this is the future of assassinations in the middle east and elsewhere?
It would seem far easier to just fly it manually for the average person. There was already a case of drug cartels using a drone with explosives, and others using swarms to attack police during a urban drug raid
High-value targets have been travelling with powerful RF jamming equipment for some time now. The only decent response to the drone threat has been "jam the control signal".
The clients were either the Libyan government (or defense contractor), or the the breakaway faction (or defense contractor), but we don't know which
We don't know who it attacked
We don't know if it killed them
That's frustrating.
Anyway no serious drone introspection is going to come until non-US drones attacks US assets, and ideally because they don't like a US ideology that there is no state-side consensus on changing
A non-US drone attacking USG assets isn't going to see the USG crack down on drones. They might use it to crack down on hacking, encryption, criticism of the USG, unlicensed drones, Bitcoin, competition from China, or copyright infringement, or other things they're against. But they won't use it to crack down on things that they're in favor of, like drones.
I think it will lead to more collaborative efforts to curb drone use, extrajudicial killings against arbitrarily created categories of people such as "non-state enemy combatants"
Very akin to bans on mustard gas or nuclear proliferation
You could sell the services of one of these via Bitcoin. Have a killer drone primed and waiting, do a crypto based auction and then the drone would autonomously lift off and perform the mission.
Using a surveillance coin for a surveillance mission, a match made in heaven!
It would make more sense to do this privately and anonymously, so Monero or funds obfuscated with Tornado.cash on the Ethereum network would suffice and a computer can still react to payment received and begin operating.
For extra chaos, control the drone with a smart contract where people vote on who to kill. The person with the most votes gets offed, and then it continues down the list of targets until it reaches a threshold where we no longer consider this "aggregated, anonymous data" (Big Tech usually uses 50 data points or so).
The interesting thing about this is that nobody pulls the trigger. You don't get killed because somebody wants you dead; you get killed because lots of people want you dead. And then everybody is hidden in the mob, and not really responsible for your death; after all, if 49 other people didn't also want you dead, you'd still be alive.
If studies on diffusion of responsibility (eg. Kitty Genovese) are true, then it's likely people will be pretty free with their nominations, knowing that the person won't actually end up dead unless lots of other people also nominate them. Mass carnage is likely to result.
That's pretty much the plot of the Black Mirror episode "Hated in the Nation" [1], except instead of smart contracts it was Twitter hashtags (and there was a nasty payback).
I think it would be illegal to participate in any way purposefully even if your nomination has a minimal chance of being selected. Enforcement is another matter
Yeah, since the mid 1990s people had been predicting that anonymous electronic money would lead to the collapse of society because people would create assassination markets:
Guess what, eleven years later society has not collapsed and there are no assassination markets. Turns out that murderous psychopaths are not as commonplace as the statists would have us believe, and those that exist are not really into it for economic reasons.
Is it that much easier? Its not like a drone can randomly fly around high profile people or infrastructure without raising suspicion. People would still have to bring the drones into an area a target is know to be, and not just a drone but something military and clearly nefarious if they are caught traveling/importing.
Countries could likely do the same thing today with explosives planted at key timings/places. And drones definitely open options in this but if a country was to act this way I think they would be just as likely with non-drones.
Overall I agree this is an extremely dangerous technology for humanity. Generally anything that will allow a small number of people control a larger number, history has show this will be often abused.
Conversely, as soon as one mobster whacks a competing mobster with one of these drones, the public might decide that drones in residential areas are not okay anymore. Suicide drones don't care about registration schemes.
I think it's at least possible these drones may actually reduce collateral damage. Some combatants work amongst civilian populations, using the potential for collateral damage in their favour. Smaller, highly targeted drones could reduce the effectiveness of using civilians as shields.
Obviously some scary as hell scenarios as well, but just pointing out there might be some benefits.
I've been dreading the "softwareification" of weapons for quite a few years now. There's not a day that goes by that I don't think about how weapons can nowadays be readily and cheaply made with commodity parts and no special training or materials. Obviously the algorithms and software for this sort of stuff needs to be made, but it's far more accessible than expertise and equipment in for example metallurgy.
There was a video a couple of years back of a kid in Texas attaching a pistol to a drone and firing off a couple of shots. Stuff like this is not rare or inaccessible, and it's only a matter of time before it becomes widespread.
The "hardwareification" is also troubling. When there is no human cost for one side to wage war, it becomes a lot more politically palatable because you no longer have family members coming home in boxes.
It's easy to say "Well, the other side are bad dudes who have it coming", but even if they are genuinely bad dudes (and not just politically inconvenient) there's no war without collateral damage (a term invented by the US during Vietnam, when they were doing a lot of it).
Vietnam was not special because of the amount of collateral damage done but rather due to some serious strides being made towards avoiding or minimizing it for the first time. Let's not forget that in Korea levelling every city north of the front line was deemed an acceptable strategy.
> I've been dreading the "softwareification" of weapons for quite a few years now. There's not a day that goes by that I don't think about how weapons can nowadays be readily and cheaply made with commodity parts and no special training or materials.
It's not so much software that's the issue, it's people taking existing tech and weaponizing it.
Software has been in weapon systems since the 1970's from guidance to target acquisition. It's already there, and has been for a while. Machine learning is just another iteration on what's already out there.
The world knows how to kill en masse for a long time now with NBC (Nuclear/Biological/Chemical) weapons -- and software wasn't a part of Fat Man or Little Boy.
But consider the challenges the US had in the middle east for the past two decades.
* Consumer drones dropping hand grenades.
* IED's using manual triggers.
* Trucks loaded with explosives blitzing checkpoints.
* Cars with explosives driving up next to convoys.
* Land mines.
You don't really need custom software, just take existing tech and get creative with it.
By "softwareization" I mean the transfer of the difficulty of making weapons from software to hardware. Programmers and programming knowledge is far more widely available that specialized hardware or techniques. We've already seen this process in many fields of business with great success, so it's no wonder that it would come to warfare as well.
And I don't see how that changes anything. I just see this as a distinction without a difference.
People have written software to target buildings, soldiers, tanks, planes, missiles, etc. This is done via radar, heat signatures, cameras -- and software already. It's not such a big change really.
I believe the thing you really have to worry about is the proliferation of NBC. It's easier to drop a nerve agent in a war zone than it is to figure out how to make an explody drone to kill your enemies and not you, say.
> Software has been in weapon systems since the 1970's from guidance to target acquisition. It's already there, and has been for a while.
Obviously correct, so in a sense the parent commenters concerns are not about something new.
But historically these weapons systems are massively expensive custom systems developed by defence contractors for major states. Costs, budget cycles, and technical limitations went some way to limiting their ubiquity and the rate they could gain functions.
So cruse missiles had autonomous terrain-matching terminal guidance, but the US needed the combined abilities of the NRO and General Dynamics / McDonnell Douglas /
Raytheon etc to build it.
Whereas now we have free and triviably cloneable software (embedded OSs, drone flight and guidance s/w, ML and image recognition stacks, global mapping data) and a massive range (compared to event ten years ago) of dirt-cheap cots hardware (GPS, lidar, ...)for making autonomous moving objects. And often those devices are fairly thin hardware shells around yet more software.
The network effects driving these capabilities, particularly in software, and their seemingly exponential increase, are what I took to be the source of the parent commenters concerns.
> Whereas now we have free and triviably cloneable software (embedded OSs, drone flight and guidance s/w, ML and image recognition stacks, global mapping data) and a massive range (compared to event ten years ago) of dirt-cheap cots hardware...
If you look at the current tactics of insurgency, software is not a part of it. Enemies of the US (Al Qaeda, etc) already had tremendous success with the current insurgency tactics (IEDs, etc) which were low tech solutions, and were vastly hard for the US to overcome.
Why would they go to software when there's plenty of tried and tested low-tech solutions already to kill your enemies?
I just think about the history of war, and it was already pretty brutal without software. I don't see how software changes that equation. More brutal than what?
It's the same multiplying effect that software has on other business ventures. Software can be scaled, optimized, perform complex features, and duplicated without cost.
Which makes me wonder what happened with the whole 3D printing guns story...at one point few years back it was front page news for a while. Weird how no one talks about it anymore...
Perhaps it's just so easy to buy guns on the black market that there's no point going through the effort of 3D printing one? I'm just speculating, I have no idea.
It think it’s more relevant that 3D printed guns mostly don’t work well and fall apart after a most a couple of shots. By the time you build something that works well you may as well get proper machining tools and make it out of metal.
I think "commoditization" might be a better word for it. In the past, the general citizenry was restricted to use weapons provided through very specific channels, and if they wanted to build their own weapons they would either be rather primitive, like bows or slings, or one would need special knowledge, tools, and/or materials.
Sure someone might have had the ability to make their own gun at home, but today there are far more people who are able to build lethal weaponry from consumer-grade materials.
Making stuff like this still requires specialized software, but software development is far more readily available and accessible. The proliferation of software is also essentially effortless, so a terrorist group or similar could have a single team supplying standardized designs worldwide.
Consider the hobby of FPV drone racing. The designs and plans are widely available, and you can get a top-notch drone for about $2000 for a first-time purchase (less for the subsequent ones, since you'll already have goggles/radios). These drones are regularly built by teenagers and it's not difficult at all to add capabilities to the drones, such as target tracking, autonomous operation, software-defined radios for jamming resistance, etc.
> “The Kargu-2 is fitted with an explosive charge and the drone can be directed at a target in a kamikaze attack, detonating on impact.”
> The drones were operating in a “highly effective” autonomous mode that required no human controller
I think "fully autonomously" is a bit of a stretch for this particular story. I'm falling to see the difference between that and a heat tracking device or a laser guided missile except for the sophistication of the target seeking system. In this case a human designated a target and pulled the trigger, as opposed to an autonomous decision to attack humans.
But that it is clear that we are at only an inch of having drones creating no mans lands on demand.
The autonomous part is troubling. People have been using drones to attack other people for over 5 years at least (I remember reading reports about the Taliban using drones in Afghanistan in 2015).
The video on the STM site is kind of lame (it shows a drone detonating a cluster munition after homing in on a high contrast target with humans spaced around it. That isn't to say that is the limit of its capabilities but it doesn't inspire confidence either.
All in all this has been a surprise to exactly nobody however. Given the cameras these things use are easily damaged by only moderately high powered lasers (like the ones used in laser light shows at rock concerts) I expect we'll see optical counter measures fairly quickly. Then there is the whole "emp gun" stuff that was being shown at the UAV conference in Las Vegas in 2018 to consider as well.
All in all a sad step toward more varied ways of killing each other. Sigh.
Am I the only person not troubled by autonomous weapons?
I’m more troubled by the way the military turns people into order following machines (an extreme exaggeration), than the fact that it had actual machines that follow orders.
As long as there’s a clear chain of command (responsibility and accountability), it’s not that different.
It will probably be like stock trading algorithms: a slightly more good than bad will be produced (more stable markets), but not a huge game changer.
Perhaps you are assuming some kind of inherent humananity in the chain of command that doesn't always exist. The further away from the conflict the easier it is to make inhumane decisions that serve your own betterment. Remember that it's not just your country and leader that gets the drones.
Human operated drones have long been criticized for dehumanizing conflict which allows for escalating harm and harming unnecessary targets without issue.
Similarly war atrocities are somewhat limited to what you can convince your army to do. Not so with drones, they won't revolt and they won't tell the news either.
Perhaps it would be nice if conflict was just drones versus drones, but that is unlikely. War doesn't happen where you all arrive to battle in a field somewhere, it will be drones flooding towns and human combatants and civilians all in the mix.
I mean I literally just said I’m worried about military turning humans into machines, so I don’t know why you’d think I see the chain of command as “humanized.”
However, I do believe the US military has a very strong “culture of accountability” - much more so than US police, US politicians, US civil servants, US scholars, or anything else in the US.
I suppose autonomous weapons might not share that culture, but whoever is deploying them does.
It’s worth pointing out that for a long time, drone strikes were the provenance of the CIA which lacks a formal chain of command.
As long as the decision to deploy an autonomous weapon, and what’s its parameters are, is set by an accountable agent, I don’t see a ton of difference between the uncertainty in an autonomous weapon, and the uncertainty present in the chaos of war.
See: US bombing of Chinese Embassy in Belgrade.
Honestly, an autonomous weapon might be much more like to avoid non-military targets (like the Chinese consulate). If so, great!
Edit. I reread your comment. Regarding atrocities, automous weapons can care about atrocities far more than non-autonomous weapons. A nuclear bomb or stick of dynamite doesn’t care who it’s blowing up. An intelligent targeting system does.
A properly trained AI, with a secure audit log, could serve as a strong deterrent against violating the laws of war.
How is this different than a missile that is programmed to hit a specific target and computer-guided to its target. We don't tell to call missiles drones, while quadcopters are, but in essence there's not much difference, and the former has existed for a long time.
209 comments
[ 4.0 ms ] story [ 244 ms ] threadI can't be the only one.
Please tell me I'm not.
They are now producing knockoff's of said drone.
Edit: 69 was a typo, but I'll keep it.
And yeah, as I wrote my comment I also envisioned self-recharging drones as a logical next step. If you're going fully autonomous, why keep any requirement to come back to base? The one in this story was kamikaze, but if you needed something with more of a punch that became expensive enough to require reusability, then you could have it come home, or just station autonomous ammunition stations at strategic locations.
https://en.wikipedia.org/wiki/Mark_60_CAPTOR?wprov=sfla1
These fall to the ground after 30 mins.
Unexploded bombs also bury themselves.
These will be attractive to children (boys) but since they can only go 5 km the children are already part of the battle.
Sure I'd add a further mode to land early and have a low power sensor to blow up when it detects a person in some cases but there's little reason to think this story is even true yet.
Unexploded ordinances predominantly kill young boys.
Even though it saves lives how we direct education campaigns, Twitter won't allow us to discuss this? What a world we live in, it's honestly falling apart.
"the effects of unexploded ordnance in Afghanistan. In this case, 81% of those injured were civilians, 92% were men and boys, and 46% were younger than 16" - https://reliefweb.int/report/world/differing-effects-explosi...
> child labor is limited.
For labour it's a farming and digging issue for adults but for the children killed it's mostly about play I'm not sure where this is coming from?
As I said given Kargu-2 are going to be on the surface or returned to base they will be quickly cleaned up. Also unless they have gone to suicide mode and missed they should be relatively harmless.
Without added programming they are not like mines.
> "The lethal autonomous weapons systems were programmed to attack targets without requiring data connectivity between the operator and the munition: in effect, a true 'fire, forget and find' capability," the report from the UN Security Council's Panel of Experts on Libya said.
I’m not sure why this is called “rogue” — apparently it was operating as intended
In this case, it seems the drone was supposed to do exactly that, and that's perhaps far more dangerous -- this is the first kill where the drone was approved to determine kill targets, and execute that decision, with no human to verify. This really shouldn't be a goal we have in the first place -- to remove humans from the decision loop -- and yet apparently here it is.
https://en.wikipedia.org/wiki/Mark_60_CAPTOR?wprov=sfla1
With this you’re creating a no-go zone for your enemies, and very likely safe for your allies (who you can tag properly with markers), but the difference between enemy and neutral is rather unclear (especially in the Middle East, where the line between combatant and civilian is very thin, almost indistinguishable even to humans on-site)
Just shutdown the grid worldwide for one month. 90% of the population will die.
The best realistic option here, in my opinion, is to require sensible constraints. E.g. hard-wiring war zones in them - a separate circuit could prevent firing if the drone is outside certain area. You could decrease the attack surface for such system by relying on a Kalman filter getting it's data from a dedicated accelerometer and a compass.
Of course most recently Syria is a horrible exception to this track record, and it remains to be seen if this will embolden others to do the same. Nonetheless, the prohibition does show that such agreements can at least reduce the usage of certain methods of waging war.
But I don't think hardware or software constraints be effective: Once one side of a conflict decides to break the a prohibition about using autonomous drones, I don't think they will care about hardware protocols. There's also the fact that warfare generally does not have static boundaries and no one willing to use this type of weapon would concede to their uselessness once the fighting moved elsewhere. Especially because their enemies would simply & quickly change their tactics to work outside that zone.
I don't disagree with your point, but this part highlights how hard this is in practice. When does "the practice end"? When the war is one? When the current offensive ends? When we believe the devices aren't being produced anymore? This immediately reminds me of how Chamberlain treated Hitler.
You can also go after the people behind the orders to do these things and bring them up on war crimes. I find the former option much more ethically reliable, as the later frequently suffers from the fact that war crimes tribunals are generally not convened against the actions of whoever wins, while the winner is all to eager to punish their enemies even beyond defeat.
But none of these options are perfect. We live in a world where bad people can do awful things and never suffer the consequences. Justice isn't dead, but it has never been equally applied.
Not completely, and for what is, in even human time-scales, not at all a long time. We'll see where it goes, but I'm not hopeful in the long run.
Regardless of what comes after this, we can say that the prohibition worked reasonably well for at least 100 years. I'd certainly be happy if we got even 25 more years before fully autonomous lethal drones became common place, and a prohibition on such weapons could buy us so e time on the issue.
Chemical weapons are not used because they were ineffective even during WW1 and make even less sense now, when there is no trench warfare and targets are generally highly mobile. Conventions have little to do with this.
Certainly they were effective enough that Japan used them in thousands of attacks against Chinese forces. Effective enough that the US and Russia and probably others actively researched and stockpiled them for decades into the cold war. They were considered a significant threat by the US during the Gulf War: Iraq had a history of using them against Iran so the US took many precautionary steps to protect or inoculate their soldiers against them. Too many precautions as it turns out because those protection methods are themselves credited as a major factor in Gulf War Syndrome which impacts hundreds of thousands of US soldiers. The US certain believed in their efficacy 70 years after the trench warfare of WW One.
I don't know where you get the idea that they are not effective, or only effective under conditions if trench warfare. Actual use shows this is not the case.
[1] https://www.youtube.com/watch?v=9CO6M2HsoIA
[2] https://en.wikipedia.org/wiki/Slaughterbots
> One way of looking at human creativity is as a process of pulling balls out of a giant urn. The balls represent possible ideas, discoveries, technological inventions. Over the course of history, we have extracted a great many balls – mostly white (beneficial) but also various shades of gray (moderately harmful ones and mixed blessings). [..] What we haven’t extracted, so far, is a black ball: a technology that invariably or by default destroys the civilization that invents it. The reason is not that we have been particularly careful or wise in our technology policy. We have just been lucky.
https://nickbostrom.com/papers/vulnerable.pdf
Try to pull one out and you get a web of affordances and side effects and unforeseen interactions stuck to it
If we were to be wiped out by our technology I see no guarantee some extradimensional coroner would be able to point definitively to just one
Say it's climate heating, rapid-onset Venus mode. Was it the internal combustion engine? The coal plant? The automobile? The wheel? Language? Money? Industrialism? The limited liability corporation? How do you pick one?
No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms
Also some balls (e.g. nuclear weapons, stray asteroids) make it a lot easier to point to something immediate and mostly unambiguous
I agree the analogy isn't perfect, but one can say that despite technologies not having a discreet origin, we can reason about them in retrospect. So maybe modern artificial neural networks were initially made possible by improvements in microscopy and experiments in cell culture to deduce the mechanisms of biological neurons, but we can reason about ANNs as a technology that is changing what is possible by a machine.
I imagine Nick would also be inclined to be fatalistic about technological progress, in that these "black balls" will inevitably be found. His main stance is that we should be enacting the global ethical frameworks around these technologies much quicker.
Yes, agreed, good point
I also agree about Nick's probable fatalism, and I think his work is brilliant and valuable, however one thing irks me. He's made a successful career out of hypothetical "black ball" speculation while saying little about the potential for blackness of the tech we already have. He frames existential risk as if we are on the cusp of dramatic danger but still relatively safe. In truth we are already over the cliff & falling rapidly
Take his paperclip maximizer. They already exist! We need not wait for some future AI to realize a badly misaligned objective function. We're already determined to transmute the remains of our homeworld's long dead into plastic and atmospheric carbon
That's a black ball.
A very popular story but it was also primarily 1 of the 8 scientists working on the Manhattan project and only because he hadn't had all the math showing how incredibly unlikely it would be shared with him. It was already a settled "No, not going to happen." before the first test was conducted. The story ends up being a popular soundbite but with no real teeth on it. [0]
[0] https://blogs.scientificamerican.com/cross-check/bethe-telle...
Oddly, this is kinda freeing in a nihilistic way. If you know that you and your species are going to die eventually (which you do, the eventual fate of the universe is heat death), you can focus on living for the present instead of brooding on the inevitable death of mankind, and just be grateful that you and your species happen to be alive at this particular time on this particular planet where life is actually a thing.
Space is big. Earth is old. When all is told, we just don't know.
"THEY sent A SLAMHOUND on Turner’s trail in New Delhi, slotted it to his pheromones and the color of his hair. It caught up with him on a street called Chandni Chauk and came scrambling for his rented BMW through a forest of bare brown legs and pedicab tires. Its core was a kilogram of recrystallized hexogene and flaked TNT."
So they will enable waging war in situations where it was previously impossible on humanitarian grounds.
Nice selling point.
Fuck that.
I honestly can't decide which is worse: Weapons so terrible nobody wants to go to war, or weapons so neat that war is the obvious choice.
Unfortunately I don't really think this matters much in war time.
Unfortunately, I'd have to put my money on munitions like these being used more often for targeted assassinations of <insert name of prominent person here>. It seems on the surface, very effective. And I could see where it would be hard, if not impossible, to trace. I think people in the future will think twice before running for mayor or agreeing to be a sheriff or police chief for instance.
For example, once the cartels build some of these, I'm not sure what kind of odds I would give anti-cartel leaders. Probably would be even worse in the US given our acrimonious dichotomy. I could see tit for tat assassinations going on all the time.
Of course, all that also means that a lot of work will likely be put into countermeasures. So who knows how all this will develop I guess?
Now, what can oppose those drones, and in particular by the probable opponents in the region? Manned air fighters, other drones, and jamming. In all those cases to preserve and maintain superiority the drones have to get fully or at least partially autonomous. Note that the Turkey isn't participating in the stealth race even though they could have gotten F-35 if they behaved, and the countries like Azerbaijan and Ukraine for example have no chances of getting stealth, so drones is the [only] way here, and autonomous drones is just the next step in the development of those weapons what their national security (or regional superpower status in case of Turkey) will be relying upon.
I'd almost prefer that this showed something like someone's head getting blown off their body, to demonstrate the actual brutality of what it is designed to do, over making it seem like some video game.
They're a really twisted group of people that mostly feel like they have no other opportunities (that maintain their lifestyle). Not so different than people employed in any other industry, just important to understand.
As hn_throwaway_99 mentions below, the "Slaughterbots" video is not too far off from how STM presents their equipment: https://www.youtube.com/watch?v=9fa9lVwHHqg.
Had a similar reaction once before, after randomly walking in a military exhibition. While I understand the need for defence capabilities, and I can be impressed by the engineering, I don't get the triumphant glitter surrounding the showcase of killing machines.
You know, the other day I saw a news headline about one of the world's biggest arms dealers being arrested... and the first thought through my mind was, "What? They arrested the United States?"
Being a small fry, independent arms deal is grounds for imprisonment, but doing it as part of a nation-wide effort will get you wealth, medals, social status and government funding.
If you’re wondering why it’s an effective video, it’s because it makes me want one. I don’t even have anything to blow up. That’s the essence of effective marketing.
We can also recognize that in the wrong hands, it’s an evil device. But so is a gun. And I would imagine the reactions to the first portable guns may have been similar.
Lots of people find guns cool, and I have lots of fond memories of shooting with my dad and a friend. Learned about gun safety and how to properly care for one. I also learned I wasn’t responsible enough to keep one, so I’ve never bought one.
This is obviously different. But it’s not inconceivable that in the future, you might be able to deploy a sentry quadcopter designed to chase down (not destroy) someone who just attacked you. I was shocked that there was a drive-by shooting in my neighborhood a few months ago. Would’ve totally sic’d a drone on them the moment I heard the gunshots.
Dunno. Sometimes I feel lonely for feeling this way, when the only acceptable social response is outrage or indignation. It’s cool to blow things up, and humans have had a long and proud tradition of making things go boom.
The key is to make sure that this tech won’t have unforeseen consequences. But as far as I can tell, the consequences appear to be foreseen in this case. It’s a drone that goes boom when next to someone.
All that said, I agree that it’s a terrible idea to let it decide to do this automatically. But in a warzone, it’s not impossible to imagine some heuristics that might make sense in certain highly constrained circumstances.
Of course, most dictators don’t really care about minimizing civilian casualties, and dictatorships seem like the target audience. But, ditto for guns.
Certainly long, but I'm not particularly proud of the many ways we've found to kill each other.
The immediate second order effect is, people start buying drones programmed to destroy a drone seeking them out. This should be much less controversial than the first drone, since it's not operating on a human target. Now the first drone needs to be able to eliminate one -- and shortly, multiple -- drone(s) before then destroying the human target. The defending drone(s) will certainly increase in number and firepower. It's basically a straight line to automated warfare in the streets with no regard for collateral damage.
The hurdles, of course, would be regulation on the ability to purchase drones, and the permitted level of firepower. I don't like the idea of us negotiating the laws around that while drones are causing collateral damage in an urban (or any) environment.
This is literally about killing another human being and you went with “Well yeah but”.
Tripwires attached to a claymore fall under boobytrapping laws under arguments of reckless endangerment. I've heard it reasoned as there's no mind behind the assault and so it's reckless even if it harms someone breaking and entering.
Facial recognition gets closer to an intelligence behind the weapon, but a less than human intelligence (less than canine?). At what point does your "tripwire" have enough intelligence to be legitimate under castle doctrine?
It also kind of has to explode itself to do it’s job, which feels like a waste to me. So many things you can do with a drone that are immediately useless if it blows itself up.
I'm not sure what your point is here? The 'but' implies that we all now agree that guns are basically OK which is not universally true, and I would say increasingly the opposite.
Also your conception of 'the wrong hands' and other peoples' are not necessarily the same - one man's terrorist is another man's freedom fighter, and so on.
> But it’s not inconceivable that in the future, you might be able to deploy a sentry quadcopter designed to chase down (not destroy) someone who just attacked you. I was shocked that there was a drive-by shooting in my neighborhood a few months ago. Would’ve totally sic’d a drone on them the moment I heard the gunshots.
I think this is exactly what scares people - there are people who would sic killer drones on people Judge Dredd style, out of a misguided sense of moral duty; I heard about a recent case of someone using the 'stand your ground' laws to justify (and be celebrated for!) going to a neighbour's house during a robbery and shooting the thieves in the back, killing them both. Not to mention that in a future where these things are available, that drive-by would've instead been a fly-by killing and you might never know who did it.
You talk about these people like their lives are very important, and that it's both worthy and honorable to protect them.
In reality, this is what it's like: https://twitter.com/theshawwn/status/1301407648420593664
It was scary as hell to have a drive-by happen. The cops couldn't do anything. They were long gone by the time the police came. And then they came back and shot up the same place again a few hours later.
If I had a drone that could track them, they may have been brought to justice. I don't see this tech the same way you do. I see it as a stepping stone towards a world where people can protect themselves and their loved ones, as they do with guns.
I think we'll just need to agree to disagree. I have a lot of things I could say, but I just don't get how people can calmly live in a country that was earned through bloodshed, while decrying those who were willing to lay down their lives for you.
This is a military device. It's designed to kill people, because this is one of life's unfortunate necessities. I think the people here who are saying that there's nothing honorable about killing someone should teleport themselves back to Midway in 1942 and tell your great-grandparents that, face to face.
> You talk about these people like their lives are very important, and that it's both worthy and honorable to protect them.
> In reality, this is what it's like: https://twitter.com/theshawwn/status/1301407648420593664
> It was scary as hell to have a drive-by happen. The cops couldn't do anything. They were long gone by the time the police came. And then they came back and shot up the same place again a few hours later.
I don't personally care so much about the lives of people who engage in this kind of behaviour, but I do care about living in a peaceful society where violence is regulated by the law rather than being handed out vigilante-style. Arming everybody does not prevent gun violence, it enables it. You need only look at the blatant correlation between the number of guns per capita in the US and the number of shootings in the US. I'm in the UK - you know what we don't have lots of? Guns. You know what we also don't have lots of? Gun violence. It still happens, of course, but it's far rarer, and I don't feel the need to arm myself just to stay safe. That drive-by you were unfortunately so close to was made easier by increased access to weapons.
> If I had a drone that could track them, they may have been brought to justice. I don't see this tech the same way you do. I see it as a stepping stone towards a world where people can protect themselves and their loved ones, as they do with guns.
The trouble is, the easier it is for you to get hold of this kind of drone tech, the easier it is for everyone, which obviously includes those who would do you harm. See my comment above. You want a drone that can follow people around? So do a host of spurned lovers and over-protective parents and so on and so on - it's not the clear win for society that you seem to think it is.
> ...I just don't get how people can calmly live in a country that was earned through bloodshed, while decrying those who were willing to lay down their lives for you.
You seem to think I'm ragging on the military, but I'm not - perhaps you're referring to other comments? In general, one's neighbours shouldn't be laying down their lives for each other on a day to day basis unless they're in the military or the police, who are, at least in theory, trained and regulated in their use of force and when to deploy it (yes there are obvious potential problems there too, but that's a different discussion).
I would also like to point out that the bloodshed is supposed to be temporary - once the wars are won and safety is assured (I'll assume you're referring to the World Wars here), we're supposed to stop shooting each other.
> This is a military device.
So are AR-15s - how many of those are in the hands of twitchy civilian militia groups and other randomers with a grudge?
> I think the people here who are saying that there's nothing honorable about killing someone should teleport themselves back to Midway in 1942 and tell your great-grandparents that, face to face.
I'm willing to bet that the majority of those who served in World War II didn't see anything honourable about it either, just that it was a grim necessity, which it absolutely was. Honour and glory are concepts the armed forces use to encourage people to do necessary but horrible things - we should of course be eternally grateful for and respectful of their sacrifices, but I think it's actually disrespectful to the fallen to glorify war, as they gave everything so their descendents wouldn't need to engage in it.
In this case, you feel that war is a grim necessity, and also admit that this is a military device. If your country is in a war, wouldn’t you want to be on the winning side? Culture often doesn’t survive losing a war.
If so, then I don’t understand why people are against development of this tech. If you’re on the battlefield, you bet that you’d want a way to take out the people shooting at you from the hill on the other side. Usually that means calling in air support. Here, your air support can be almost pocket sized.
It’s pretty much as simple as that, and I think the crux of the disagreements. People seem worried that this tech will get out of hand. I am too. But it also feels inevitable.
Would you want to fight a war without guns? Because it sounds similar to an argument against developing guns.
What happens when people who did the driveby get the technology (if you can buy the drones so can the bad guys).
What happens when your autonomous drone decides to shoots neigburs kid for playing hid and seek, in a fashion that drone algorithms find objectionable.
And suppose your drone goes after them, finds them, then what ? You do realize that after, the drive by is done and they "flee" the scene, you can't just go after them and shoot them as a civilian? You can defend your property (someplaces ) but that does not include chasing after them.
Is the drone smart enough not to shoot when there are other people behind it's targets ? what if it's Halloween ?
Supose those drones get sold, and given current IOT security standards, someone hacks them ?
I don’t like it either, for what it’s worth.
This scares the shit out of me. Call me naive but the idea that somebody on the other side of the planet can set me as a target in a device that they just set and release is completely inhuman. If you want me dead, then do me the curtesy of at least putting your own arse on the line to achieve it.
The potential for this type of device to be cobbled together is well within the capability of pretty much any reasonably funded electronics hobbyist, and I think one of the reasons that we haven't seen it more frequently is that hitherto it's not been morally acceptable. If this type of weapon becomes mainstream in government arsenals then you can kiss that goodbye.
https://www.youtube.com/watch?v=K0om03oXX_8
Awesome launching system.
We still haven't learnt to live with one another - after all these millennia on earth.
The original source looks to be https://www.newscientist.com/article/2278852-drones-may-have..., but that's hardwalled and so off topic here (https://news.ycombinator.com/newsfaq.html, https://news.ycombinator.com/item?id=10178989). If there's a better accessible source, we can change it again. I've used the NS headline since it's the most neutral.
The first self-guided antiship missiles might also qualify, but they were (iirc) lock-on before launch weapons. (Operational in the Pacific theater during WW2, even!)
The clients were either the Libyan government (or defense contractor), or the the breakaway faction (or defense contractor), but we don't know which
We don't know who it attacked
We don't know if it killed them
That's frustrating.
Anyway no serious drone introspection is going to come until non-US drones attacks US assets, and ideally because they don't like a US ideology that there is no state-side consensus on changing
Very akin to bans on mustard gas or nuclear proliferation
You might believe in an omnipotent state that will close all the markets and go after node operators making them cease.
I believe that transactions will continue and drones will continue blowing up behind people's heads.
With drones!
It would make more sense to do this privately and anonymously, so Monero or funds obfuscated with Tornado.cash on the Ethereum network would suffice and a computer can still react to payment received and begin operating.
The interesting thing about this is that nobody pulls the trigger. You don't get killed because somebody wants you dead; you get killed because lots of people want you dead. And then everybody is hidden in the mob, and not really responsible for your death; after all, if 49 other people didn't also want you dead, you'd still be alive.
If studies on diffusion of responsibility (eg. Kitty Genovese) are true, then it's likely people will be pretty free with their nominations, knowing that the person won't actually end up dead unless lots of other people also nominate them. Mass carnage is likely to result.
1. https://en.wikipedia.org/wiki/Hated_in_the_Nation
https://en.m.wikipedia.org/wiki/Assassination_market
Guess what, eleven years later society has not collapsed and there are no assassination markets. Turns out that murderous psychopaths are not as commonplace as the statists would have us believe, and those that exist are not really into it for economic reasons.
Any number of countries with animosity against each other now can attempt to neutralize their opponent without much effort.
Before there a deterrent was to invade and overcome you had to amass quite a bit of materiel, supply chain as well as have competent leadership.
now all you need is a plan and some drones and you can strategically weaken an opponent.
Countries could likely do the same thing today with explosives planted at key timings/places. And drones definitely open options in this but if a country was to act this way I think they would be just as likely with non-drones.
Overall I agree this is an extremely dangerous technology for humanity. Generally anything that will allow a small number of people control a larger number, history has show this will be often abused.
This may not remain true for long, if food and package delivery via drone become commonplace.
Many have feared 3d printed weapons, but it has been years since we have seen functional ones and nothing changed.
Obviously some scary as hell scenarios as well, but just pointing out there might be some benefits.
There was a video a couple of years back of a kid in Texas attaching a pistol to a drone and firing off a couple of shots. Stuff like this is not rare or inaccessible, and it's only a matter of time before it becomes widespread.
It's easy to say "Well, the other side are bad dudes who have it coming", but even if they are genuinely bad dudes (and not just politically inconvenient) there's no war without collateral damage (a term invented by the US during Vietnam, when they were doing a lot of it).
It's not so much software that's the issue, it's people taking existing tech and weaponizing it.
Software has been in weapon systems since the 1970's from guidance to target acquisition. It's already there, and has been for a while. Machine learning is just another iteration on what's already out there.
The world knows how to kill en masse for a long time now with NBC (Nuclear/Biological/Chemical) weapons -- and software wasn't a part of Fat Man or Little Boy.
But consider the challenges the US had in the middle east for the past two decades.
* Consumer drones dropping hand grenades.
* IED's using manual triggers.
* Trucks loaded with explosives blitzing checkpoints.
* Cars with explosives driving up next to convoys.
* Land mines.
You don't really need custom software, just take existing tech and get creative with it.
People have written software to target buildings, soldiers, tanks, planes, missiles, etc. This is done via radar, heat signatures, cameras -- and software already. It's not such a big change really.
I believe the thing you really have to worry about is the proliferation of NBC. It's easier to drop a nerve agent in a war zone than it is to figure out how to make an explody drone to kill your enemies and not you, say.
Obviously correct, so in a sense the parent commenters concerns are not about something new.
But historically these weapons systems are massively expensive custom systems developed by defence contractors for major states. Costs, budget cycles, and technical limitations went some way to limiting their ubiquity and the rate they could gain functions.
So cruse missiles had autonomous terrain-matching terminal guidance, but the US needed the combined abilities of the NRO and General Dynamics / McDonnell Douglas / Raytheon etc to build it.
Whereas now we have free and triviably cloneable software (embedded OSs, drone flight and guidance s/w, ML and image recognition stacks, global mapping data) and a massive range (compared to event ten years ago) of dirt-cheap cots hardware (GPS, lidar, ...)for making autonomous moving objects. And often those devices are fairly thin hardware shells around yet more software.
The network effects driving these capabilities, particularly in software, and their seemingly exponential increase, are what I took to be the source of the parent commenters concerns.
If you look at the current tactics of insurgency, software is not a part of it. Enemies of the US (Al Qaeda, etc) already had tremendous success with the current insurgency tactics (IEDs, etc) which were low tech solutions, and were vastly hard for the US to overcome.
Why would they go to software when there's plenty of tried and tested low-tech solutions already to kill your enemies?
To kill more people, whilst taking even lower risks themselves.
They probably wouldn't write the software themselves -- they'd get it from e.g. some supportive dictatorship
(With "them", I'm personally not thinking about the talibans, but other groups)
Sarin gas? Nuclear weapons? Truck Bombs? Strapping bombs onto children?
Cheaper and more accessible though. Not many people can build or buy their own nukes or sarin gas.
Sure someone might have had the ability to make their own gun at home, but today there are far more people who are able to build lethal weaponry from consumer-grade materials.
It's a frightening development.
Consider the hobby of FPV drone racing. The designs and plans are widely available, and you can get a top-notch drone for about $2000 for a first-time purchase (less for the subsequent ones, since you'll already have goggles/radios). These drones are regularly built by teenagers and it's not difficult at all to add capabilities to the drones, such as target tracking, autonomous operation, software-defined radios for jamming resistance, etc.
> The drones were operating in a “highly effective” autonomous mode that required no human controller
I think "fully autonomously" is a bit of a stretch for this particular story. I'm falling to see the difference between that and a heat tracking device or a laser guided missile except for the sophistication of the target seeking system. In this case a human designated a target and pulled the trigger, as opposed to an autonomous decision to attack humans.
But that it is clear that we are at only an inch of having drones creating no mans lands on demand.
The video on the STM site is kind of lame (it shows a drone detonating a cluster munition after homing in on a high contrast target with humans spaced around it. That isn't to say that is the limit of its capabilities but it doesn't inspire confidence either.
All in all this has been a surprise to exactly nobody however. Given the cameras these things use are easily damaged by only moderately high powered lasers (like the ones used in laser light shows at rock concerts) I expect we'll see optical counter measures fairly quickly. Then there is the whole "emp gun" stuff that was being shown at the UAV conference in Las Vegas in 2018 to consider as well.
All in all a sad step toward more varied ways of killing each other. Sigh.
I’m more troubled by the way the military turns people into order following machines (an extreme exaggeration), than the fact that it had actual machines that follow orders.
As long as there’s a clear chain of command (responsibility and accountability), it’s not that different.
It will probably be like stock trading algorithms: a slightly more good than bad will be produced (more stable markets), but not a huge game changer.
Human operated drones have long been criticized for dehumanizing conflict which allows for escalating harm and harming unnecessary targets without issue.
Similarly war atrocities are somewhat limited to what you can convince your army to do. Not so with drones, they won't revolt and they won't tell the news either.
Perhaps it would be nice if conflict was just drones versus drones, but that is unlikely. War doesn't happen where you all arrive to battle in a field somewhere, it will be drones flooding towns and human combatants and civilians all in the mix.
However, I do believe the US military has a very strong “culture of accountability” - much more so than US police, US politicians, US civil servants, US scholars, or anything else in the US.
I suppose autonomous weapons might not share that culture, but whoever is deploying them does.
It’s worth pointing out that for a long time, drone strikes were the provenance of the CIA which lacks a formal chain of command.
As long as the decision to deploy an autonomous weapon, and what’s its parameters are, is set by an accountable agent, I don’t see a ton of difference between the uncertainty in an autonomous weapon, and the uncertainty present in the chaos of war.
See: US bombing of Chinese Embassy in Belgrade.
Honestly, an autonomous weapon might be much more like to avoid non-military targets (like the Chinese consulate). If so, great!
Edit. I reread your comment. Regarding atrocities, automous weapons can care about atrocities far more than non-autonomous weapons. A nuclear bomb or stick of dynamite doesn’t care who it’s blowing up. An intelligent targeting system does.
A properly trained AI, with a secure audit log, could serve as a strong deterrent against violating the laws of war.
This drone can probably be had for < $10,000 and it can be set it off from the boot of a car by a 10 year old with minimal training.
Price.