"Kill everyone in the area" is probably the least harmful for the reason described.
It is much more dangerous when they start to be selective. Then people start trusting the selection capabilities and use them in cases where they wouldn't use a "kill everyone" weapon.
One way is that selective killing is more useful and therefore desirable. For example consider the classic slaughterbot video. It's plausible that someone capable might want to kill all democratic senators. It's much less likely that someone capable will want to kill all of them with a massive bomb.
I think the ancestor drones are land and sea mines, or really any kind of trap that dislocates the timing and control of the "trigger" from the person who launched it into the environment.
These newer drones have just gained locomotion instead of having to wait for victims to come to them.
Big "an AR 15 is just an automatic bow which is basically a spear thrower which is basically a knife which his basically a punch so nothing matters anyways" vibe
Autonomous drones are much more precise, thus safer for non-combatants, than land mines (which kills for decades after a war), shells, missiles, guided bombs, etc.
I'm not sure I'd consider a trap an "autonomous weapon". The trap cannot select a target. It will go off for anyone unlucky enough to step in its trigger.
An autonomous drone will select a target and pull the trigger. It fills in the position of a human pulling a trigger, which is a decision.
Maybe if robots began deciding where to lay mines, i could hand it to you.
Norbert Wiener’s cybernetics was based on automated killing — which he beautifully disavowed in peacetime [1]. Which historically is one of the main reasons we think about “Artificial Intelligence” instead of cybernetics (Wiener kind of pissed off the defense dept).
> Carpet bombing of cities, towns, villages or other areas containing a concentration of protected civilians has been considered a war crime since 1977, through Article 51 of Additional Protocol I of the Geneva Conventions.
Autonomous weapons have hardly been deployed yet, maybe at the Inner Korean border or some lunatic's backyard. Therefore I don't think there is any legislation for it yet. But it seems a very cruel way of killing, also considering in this particular case they didn't even send footage back. What kind of experiment was this? Maybe they didn't like to see the brutality, perhaps people begging for mercy not to be killed, giving up and showing a white flag. Indeed this isn't possible with carpet-bombing.
First, I advise a modicum of skepticism to be retained in the face of such news. Ukraine is, after all, in the middle of an existential crisis and must take every advantage it can, even if it's just scaring Russian invaders further (I bet both Russian and Ukrainian soldiers are already pretty scared of drones).
Additionally: "“There is no connection to the drone at all, you cannot see the video, nothing…". So there's no way to know exactly what happened which adds a lot of uncertainty.
Finally: the system was first used two years ago once, then never again. That doesn't sound like it's giving much of an advantage. Sorry, I don't believe that it's a matter of military ethics. If Ukraine could deploy actual Terminator robots to the front line it would do it in a heartbeat. Again: existential crisis. They're fighting for their country's existence. I would use every weapon in my disposal; and I'm a pacifist who hates violence. So I don't think that "test" really worked well at all.
Now, taking the New Scientist's reportage at face value, the announcement seems to describe a system that is only marginally more capable than a self-guided missile. It seems that a quadcopter swarm of undisclosed strength flew to a predetermined location (nothing new to see here), then a target acquisition system was activated.
Is the latter a new capability? Hard to say without more details that we're not likely to know. Maybe the drones simply locked on to whatever moved. Motion sensing is not new technology. Nor is it a great idea to put it on a flying grenade that you fire-and-forget.
Maybe the drones had some on-board machine vision system that tries to identify useful targets like persons and vehicles. That's eminently possible with modern tech, I have a Raspberry Pi-powered quadruped from China that can detect my face, identify balls of different colours etc. All this is more than enough to automate target selection, with a bit of creative cobbling together of existing components and if you don't care too much who the target selected, is.
Without more information it's very hard to guess exactly what happened. However, "Slaughterbots" these don't seem to have been.
Later, a different, human-piloted drone was sent in to inspect the outcome. Why human-piloted? Well, because there's no way to ensure that an autonomous drone will be able to do the job, that's why.
So in other words: we're not there yet. "There" being a nightmare where machines kill humans autonomously and we unlock a new level of horrors and war crimes. There is still time. We can still pull back from the brink. Resistance is not futile.
We've had loitering munitions that choose their own target autonomously for a long time, for example anti-tank weapons that climb up after being released from a plane or helicopter then sit on a parachute until spotting one or more tanks and firing warheads at them.
The superficial new thing here is the exact quadcopter form factor, but the significance is the new price point. You bet the loitering anti-tank weapon costs a fortune. These drones are very cheap.
Of course, mines can be even cheaper, but you unwittingly engage them rather than them engaging you.
This was back in 2026, before they released the Dredd series for civilian applications with onboard due processing and agents for prosecution, defense, judge, jury and executioner with millisecond response times, drastically reducing price per perpetrator.
I hate to burst the New Scientist's bubble, but this is nothing new. We have had systems for decades that operated under "Fire and Forget". We have missles that either go to a pre-designated point or chase a heat or radar signature once they are fired.
Human soldiers kill civilians and other soldiers on the same side. It is called "friendly fire". It is horrible, and should be avoided, but humans are more likely to make this kind of mistake more than a computer or AI model.
What's gonna happen when the redneck militias start building these on their compounds? I'm terrified of domestic implications - police departments can't go buy old military gear to squash these yet.
Can someone who got past the paywall tell me what the definition of “fully autonomous” is here? I’m guessing it means the software selected targets based on something the human operators decided when they launched the drone? Is the criteria geographic (any humans in a certain zone), or known individuals (facial recognition), or based on enemy uniforms/visual descriptions, or a specific behavioural rule (anyone emerging from this building), or what?
Something like a proximity mine from a video game. I always did wonder why they didn't just sprinkle a bunch of these in an intermediary zone. Surely it provides incredibly effective area denial. Perhaps it does and you can't get in, perhaps it invites escalation. Definitely seems like widespread automated hunter-seekers will be some kind of terror operation in the next quarter century. Too easy to build, too easy to deploy, and you don't even have to be around when it activates. Controlling the explosives pipeline is surely the only way. I'm surprised it hasn't already happened.
> “We just launch it and we know everything will be dead – everything that will be found there in this particular area will be dead,” says Kokhanovskyy. “There is no connection to the drone at all, you cannot see the video, nothing… Everything it sees will be killed.”
This is a clear war crime. We don’t need to update any international laws, do we? This is by definition indiscriminate, like landmines or chemical weapons.
That’s a war crime, but I have seen unhinged drone operators who prefer to fly these drones than doing anything else because “they like the killing”, exact words said by one.
Technically speaking, it’s not impressive, you can train a simple local object recognition in the companion computer in the drone to fly at that specific object, so just train it on soldiers images and communicate over serial port and let the open source auto pilot do the navigation, again, nothing is that impressive, a weekend project, personally done before but not on humans and it was for good.
The meta keeps evolving. I suppose the next step is to create drones designed to intercept each other. Then swarms of drones will battle it out in the sky fully autonomously.
48 comments
[ 2.8 ms ] story [ 46.2 ms ] threadIf you release an autonomous drone in an area, it will probably kill everyone there, but might use its AI to decide not to kill some people there.
Why is the latter worse than the former ?
It is much more dangerous when they start to be selective. Then people start trusting the selection capabilities and use them in cases where they wouldn't use a "kill everyone" weapon.
Ai is like 8yr old with gun: unpredictable
If a military can bomb you or drone you, it will bomb you and drone you.
These newer drones have just gained locomotion instead of having to wait for victims to come to them.
An autonomous drone will select a target and pull the trigger. It fills in the position of a human pulling a trigger, which is a decision.
Maybe if robots began deciding where to lay mines, i could hand it to you.
[1] “A Scientist Rebels,” 1947 http://lanl-the-back-story.blogspot.com/2013/08/a-scientist-...
Or, I can have a drone with an LPR slam a mortar round into your car as you drive.
https://en.wikipedia.org/wiki/Carpet_bombing
Autonomous weapons have hardly been deployed yet, maybe at the Inner Korean border or some lunatic's backyard. Therefore I don't think there is any legislation for it yet. But it seems a very cruel way of killing, also considering in this particular case they didn't even send footage back. What kind of experiment was this? Maybe they didn't like to see the brutality, perhaps people begging for mercy not to be killed, giving up and showing a white flag. Indeed this isn't possible with carpet-bombing.
First, I advise a modicum of skepticism to be retained in the face of such news. Ukraine is, after all, in the middle of an existential crisis and must take every advantage it can, even if it's just scaring Russian invaders further (I bet both Russian and Ukrainian soldiers are already pretty scared of drones).
Additionally: "“There is no connection to the drone at all, you cannot see the video, nothing…". So there's no way to know exactly what happened which adds a lot of uncertainty.
Finally: the system was first used two years ago once, then never again. That doesn't sound like it's giving much of an advantage. Sorry, I don't believe that it's a matter of military ethics. If Ukraine could deploy actual Terminator robots to the front line it would do it in a heartbeat. Again: existential crisis. They're fighting for their country's existence. I would use every weapon in my disposal; and I'm a pacifist who hates violence. So I don't think that "test" really worked well at all.
Now, taking the New Scientist's reportage at face value, the announcement seems to describe a system that is only marginally more capable than a self-guided missile. It seems that a quadcopter swarm of undisclosed strength flew to a predetermined location (nothing new to see here), then a target acquisition system was activated.
Is the latter a new capability? Hard to say without more details that we're not likely to know. Maybe the drones simply locked on to whatever moved. Motion sensing is not new technology. Nor is it a great idea to put it on a flying grenade that you fire-and-forget.
Maybe the drones had some on-board machine vision system that tries to identify useful targets like persons and vehicles. That's eminently possible with modern tech, I have a Raspberry Pi-powered quadruped from China that can detect my face, identify balls of different colours etc. All this is more than enough to automate target selection, with a bit of creative cobbling together of existing components and if you don't care too much who the target selected, is.
Without more information it's very hard to guess exactly what happened. However, "Slaughterbots" these don't seem to have been.
Later, a different, human-piloted drone was sent in to inspect the outcome. Why human-piloted? Well, because there's no way to ensure that an autonomous drone will be able to do the job, that's why.
So in other words: we're not there yet. "There" being a nightmare where machines kill humans autonomously and we unlock a new level of horrors and war crimes. There is still time. We can still pull back from the brink. Resistance is not futile.
The superficial new thing here is the exact quadcopter form factor, but the significance is the new price point. You bet the loitering anti-tank weapon costs a fortune. These drones are very cheap.
Of course, mines can be even cheaper, but you unwittingly engage them rather than them engaging you.
Or is this the first time a soldier was killed, all those other times being civilians?
Pokémon Go Driven Drones Autonomously Killing.
They develop consciousness and turn it around, and try to catch every human and Transfer them to the Professor for Candy.
https://news.ycombinator.com/item?id=48487029
https://dronexl.co/2026/06/09/pokemon-go-scans-niantic-vanto...
The two sides are quite evenly matched
Human soldiers kill civilians and other soldiers on the same side. It is called "friendly fire". It is horrible, and should be avoided, but humans are more likely to make this kind of mistake more than a computer or AI model.
We 'll just train some pigeons to eat the quadcopters, problem solved.
This is a clear war crime. We don’t need to update any international laws, do we? This is by definition indiscriminate, like landmines or chemical weapons.
Technically speaking, it’s not impressive, you can train a simple local object recognition in the companion computer in the drone to fly at that specific object, so just train it on soldiers images and communicate over serial port and let the open source auto pilot do the navigation, again, nothing is that impressive, a weekend project, personally done before but not on humans and it was for good.
Let's say a swarm of these things kills every civilian patient in a hospital or every child in a school. Who is responsible?
I'm surprised Israel wasn't the first to field such weapons.
For all the moralising I am sure most of us would prefer AIs battling it out to enlisting.