I would argue `s/pulls the trigger/gives the go ahead/` is intuitively more appropriate. The sailor on the boat that fired off the tomahawk (iirc) was likely not in a position to make an in-the-moment determination
Completely disagree. When we’re talking about something like air, missile, or artillery strikes, the operator of the weapon system may literally never even see the target, on a screen or otherwise. They may simply be entering targeting information provided to them by an operations center. These decisions are social and group decisions, with distributed responsibility, which is why it’s so important to have institutions like the Civilian Protection Center of Excellence which historically has identified and helped implement the policies which prevent these kinds of disasters, but which unfortunately the current administration decided to shut down without a replacement.
Sometimes, the difference between a successful strike on a lawful combatant and disaster is one analyst pointing out that the building the enemy is hiding in is next to a square that’s used for a market on Tuesdays and wouldn’t you know it, it’s Tuesday. That’s not something the person “pulling the trigger,” can always be expected to know, which is why every ops center needs to be designed to elevate those voices, when the time comes
While I think that it's important to do root cause analysis, we need to be very careful not to absolve blame from the people that decided to shift a significant weight of decision-making authority to an AI. An AI can be tried in a court. There has to be a responsible human for everyone one of its actions, especially when it's killing (-people-) children.
> The Pentagon chief last year slashed offices that didn’t contribute to his goal of “lethality,” including the group that assists in limiting risk to civilians, known as the Civilian Protection Center of Excellence. Around 200 employees who worked on the issue, including at that office, have been reduced by about 90 percent, according to two current and former officials and a person familiar with the effort. The team that handles civilian casualties at Central Command, which oversees the Middle East, has dropped from 10 to one.
Negligent homicide in pursuit of their DEI goals, basically.
>people that decided to shift a significant weight of decision-making authority to an AI
And I hedged that. Anything short of 100% responsibility with humans is a failure. I get investigating how it occurred, but that needs be very careful to separate ought from is.
Taking AI out of the equation, surely these same kinds of mistakes have been made in the recent past. As in, not doing due diligence and mistakenly killing civilians
(setting aside when civilians are purposefully harmed which is a conversation about morals rather than the capability to make correct decisions)
What I am getting at is, has AI really made the problem worse or has the mechanism just changed?
We are increasingly seeing "Sorry out <Black box> made a mistake, the developers are fixing it" rather than any one person taking responsibility for the decision. This is a growing problem that is made worse by AI. Over-inflated trust in AI output is also documented issue; "The perfect billion dollar targeting system with a 99.9% success-rate (on its verification dataset)" just made a "fluke"; "Nothing could have stopped this tragedy"
When it was just called a "database query" or "algorithmic estimation", it was more implied that the actual decision is left to be made and that no analysis had actually been done yet. But when AI preemptively says "I've analyzed every source of data and come to the conclusion that this ship is carrying nuclear bomb material"; it really sounds urgent doesn't it? It probably doubled down when asked about it.
The military leaders can't be tried in court for this either unless I'm mistaken. They have their own court system of sorts that would be used here and hold people to the standards of military code of conduct as this was an official act of duty.
DOGE strikes again. Edit actually looks like it was Hegseth himself, king Testosterone.
> Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. Centcom’s group shrank from 10 people to one.
There is zero chance this would have done anything beyond greatly embarrassing the US, because China would've understood immediately that US intelligence simply screwed up, which is no longer much of a surprise.
The bigger story there is, like here, treating AI as a 'super intelligence' instead of a token prediction algorithm which is increasingly accurate, but still prone to hallucinations. It's like how self driving vehicles can drive with human to superhuman efficiency, yet then have failure modes like driving head first into a concrete divider. This isn't the sort of stuff you want touching anything critical without significant human oversight. Granted, at least in the case you're referencing that oversight did indeed exist, but it's not clear whether that was good fortune or systemic.
That's because the Hugging Face incident wasn't intended to be an example of the real, present dangers of unchecked AI use in the real world. It was designed to demonstrate the unlimited power (and thus value) of their models, and thankfully it failed to do so.
AI labs don't stand to benefit from the tech being understood as dangerous for its lack of intelligence. They need it to be understood as dangerous like an intelligent alien civilization.
nice, Karp really reeled in a nice big fish, and ofcourse he is going to blame OpenAI/Anthropic for giving the targets, this explains his CNBC rants about the frontiers, his tool was the trigger in a killchain of hallucination.
> It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”
Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.
The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.
The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.
> “No more politically correct and overbearing rules of engagement,” - Hegseth
...
> Hegseth and the commander in charge of the Iran operation, Adm. Brad Cooper, were questioned about the strike before Congress. Hegseth said the U.S. does not target civilians and would investigate
So the administration does not intentionally target civilians, they have just stopped caring about who a civilian is.
Well Israel has defined "combatant" as "male between 14 and 55", and the Western world more or less shrugged and moved on without missing a bit. I'm not optimistic.
You might be confusing the approach for estimating combatant casualties after the fact. There were arguments made regarding the likelihood of military age casualties to actually have been combatants. That is not relevant. As far as who is considered a combatant during combat operations Israel follows international law.
EDIT: there is a lot of complexity here because Article 4 applies to soldiers (that e.g. fight in uniform) and not to organizations like Hamas and the PIJ that are considered "unlawful combatants" by Israel and fight in civilian clothing. But simply being a male of military age does not make one a combatant, lawful or legal, by Israel.
Instead of worrying about woke things like "is this person actually a combatant actively fighting" they want the much simpler rule of "everyone we kill was actually a terrorist that deserves to die".
Basically they stopped caring those woke UN rules about war crimes.
Politically correct for this type of ultra-religious zealot just means things they don't believe in. I don't believe he thinks of the children of muslims as real people. The guy is a fanatic who is, from his point of view, on a literal crusade for America.
> So the administration does not intentionally target civilians, they have just stopped caring.
FTFY. IMO I don't think they directly stopped caring about bombing civilians but have stopped caring about doing their jobs. And this staggering tragedy is the outcome.
I think with the atmosphere this current administration has created, questioning things is likely frowned upon. Better to just follow orders and if shit hits the fan, put on a circus act and move on to the next tragic disaster in the making. Someone else can always take the blame.
Now, ask yourself if you want AI taking decisions around your life... it will pull numbers... someone will have a "goal" of 1000 "hits" and there you go, 2-3 months in jail working to check what went wrong...
But don't worry, the other P AI system will ensure that the first one doesn't do mistakes...
that smells like bullshit to be fair. if you have a system that makes errors over 90% of the time then you have a system you can leverage to make fewer errors.
the situation would have to be very particular for that not to be the case.
The situation is very particular. When the system makes an error, I make a bigger profit. If the system made fewer errors, it would approve more things that cost me money.
Why do you think I'm motivated to make fewer errors under that situation?
That's not how type 1 & type 2 errors work. The article is describing type 2 (false negative) and inverting the output has no correlation with the percent of false positives (type 1)
Nah, it's cumulative errors per claim. E.g. maybe the per-decision-factor rate is somewhere over 50% but overall most claims will have errors because so many factors go into coverage/reimbursement decisions.
> if you have a system that makes errors over 90% of the time then you have a system you can leverage to make fewer errors.
member the three words "delay, deny, depose"? Insurances - and not just in the US, Germany has similarly bad stuff going on - aren't making a profit if they just blindly accept claims. It is much more profitable to deny with a vague "AI" system (or a blanket automated deny) first, and only have a first look with an actual human agent at it when the customer complains or files a lawsuit.
I can easily build you a system that will guess the price of a stock tomorrow wrong at least 90% of the time. Even 100% wrong if you'd like. How would you use that to make fewer errors in some other system?
What’s binary choice we are discussing here and what’s the opposite that you’d do? Approve every claim that was denied by the AI, and vice versa? Or what’s the binary choice related to stock prices you’re referring to?
I agree, that most real world problems, are not binary choices. But in this case actually yes, just do the opposite.
For that reason error rates above 50% don't make any sense. Building a system, that is reliably wrong, is as hard as building one that is reliable correct, because the systems are the same, just with the output inverted.
The worst you can get with a system is 50%, at which point you can just flip a coin, because that is just pure random. Any deviation from that is going to be an improvement.
I always thought insurance claims were more complex, some things may be wholly or partially covered on a line by line basis, and there should be justification provided. In certain cases they go to mediation or even court. It doesn’t seem binary to me but maybe it works differently where you are.
Judicially, you can hold a person accountable for "accidentally" killing 100+ children easier than you can a technology system. The technology system will always be used as the scapegoat for the government's atrocities.
I'm not sure why the "people make mistakes too" crowd can't seem to grasp this concept.
No, but the US has historically permitted even cold-blooded torture of innocents with no liability, and has asserted the state secrets privilege to prevent lawsuits and prosecutions, so it wouldn't have mattered.
But that's a US-specific thing. With humans you at least have this theoretical liability-- that there is someone who could be punished.
AI is a different order of magnitude since unlike all other technology across all of human history you can hand off the task and decision-making itself entirely to the AI.
It's the difference between a sword and a nuclear missile even though both are "technology"
You seem to have missed the point, which is that the decision to murder a large number of innocent people was made by humans at the top of the chain of command, and 'AI' is being used as a thin pretext for that.
The pattern matches various other acts of senseless violence against innocents in South America.
The message isn't "We did this accidentally. We should understand how it happened, so we can make sure it doesn't happen again.".
It's "We do this to make the point that we can, no one can stop us, and if we're challenged we're going to lie about it."
The decision was indeed made by AI and humans failed in their duty to fucking verify it. They green lit and now we have school children dead. No single component of this whole chain is defensible.
Depends on what "decision" means. If I say, "Claude, compile a list of 1000 buildings to blow up," and then skim the output briefly and then press the red button, I wouldn't say Claude "decided" anything.
We had a public tender for some 3d model of a railroad project a few years ago, three companies applied and for some reason (corruption) the most expensive one was selected to make the model.
They blamed excel for the bad choice. Excel.
I mean sure, you can always blame the tool, i can blame my cars rear parking sensors when i hit someone when backing up during parking, but come on... at some point, a human is responsible. Someone, some person, decided and greenlit the decision to bomb those places and should now be in jail for that.
I mean, what's next? A building collapses, because the load bearing sections are too weak... do we blame CAD tools now?
If someone made a table of one hundred potential targets and rolled ten ten sided dice to pick a row to target, do we blame the decisipn on the dice, the table, Wizards of the Coast, or the person who decided this was a good way to choose targets?
You’re conflating who chooses the decision maker with the decision making device.
So AI’s output is not so different than a roll of dice? Definitely the people who decided to put in such an incapable and unreliable tool should be held accountable. No dispute in that.
>The decision was indeed made by AI and humans failed in their duty to fucking verify it.
The decision to leave the decision to AI was made by humans. It's on the moronic humans for not understanding the limitations and flaws of the software they are using and believing the marketing material without constantly verifying their results.
> It's "We do this to make the point that we can, no one can stop us, and if we're challenged we're going to lie about it."
FWIW, forcing people to go along with transparent lies is a popular social manoeuvre. It signals both the power of a group and the power of the person leading the lie.
Although it is a very frustrating tendency, because honesty and realism tends to be better for the people who have the numbers. It is hard to organise real efforts around lies.
> Now, ask yourself if you want AI taking decisions around your life... it will pull numbers... someone will have a "goal" of 1000 "hits" and there you go
Before anyone does this please let me know beforehand so I can invest in paperclip factories…
The US government just tries to shift blame for atrocities. This also, by
the way, coincides with the US government trying to abolish a world court
investigating war crimes. Hopefully we'll have an objective world court
one day, so ALL war criminals get prosecuted, at all times.
Important context is that the context wasn’t even in a connected system. Unclear if the human analysts would have even caught this, or if they would have known to check this unconnected system.
> Some US intelligence analysts took notice, the officials said. Bloomberg reported in June that one analyst detected changes to the site as early as 2019, and logged remarks about the modifications in a system that wasn’t connected to the primary military intelligence database, which informs targeting
>"Unclear if the human analysts would have even caught this" (seizethecheese)
At the time a news outlet put out commercially purchased images of the target site before and after -- even a layman could see that it had a children's playground next to the building, and that it was separated from the compound.
In short, this seems entirely avoidable. I don't think you need any connected system, you just need an image of the target and eyes.
If they trained a model to find targets and didn't train it to avoid civilian targets, and couldn't be bothered to have anyone check them, that's still a war crime. The analyst could probably see the kids playing there with the imaging tech they have available.
This war was unlawfully started by the military at Trump's command, knowing it was not a lawful order; just as they knew firing on shipwrecked civilian sailors was murder. Seemingly it was started to win money betting on fuel prices.
The discussion is mostly about AI being evil. Humans, even the very large amounts of humans the US military employees, are fundamentally limited in capacity. So the odds of them seeing this sort of image, and canceling the targeting information are approaching a constant (and that constant is not 0)
Now if you ACTUALLY used AI, as opposed to what happened here (Trump stooge getting request for 1000 targets and doing SELECT * FROM targets LIMIT 1000, except getting AI to write the query) could could drive the odds of this happening much closer to zero.
The whole point of using the AI systems it to review vast amount of data. They would 100% have access to updated pictures if we as civs can pull that data from public sources. Its not like the school was new, it had existed for over 10 years!!
"In the final days of February, the Trump administration demanded an overwhelming aerial assault. More than 1,000 Iranian targets were hit within the first 24 hours alone. That order compressed the time frame for finalizing targets, the officials said, instilling an urgency that — combined with gaps in underlying information about the site, cuts in civilian-protection personnel and an overreliance on artificial-intelligence technology — set the stage for the deadly error."
-------------------
Maven is an untested tool for rapidly selecting targets under time pressure during a war. It's reason for existing is speed. If you're in a shooting war, the ability to select good military targets faster than your enemy, but not necessarily with zero error, is an advantage.
Planning a first strike is a different situation. Time pressure should not be a significant factor. Humans should have been doing their due diligence on every single target. If an untested tool for rapid mid-battle target selection was relied upon completely because of a hasty executive order, then the responsibility falls entirely on the executive.
I think that’s the important distinction. Mistakes under pressure do happen. Unnecessary pressure is not a mistake, though, it’s a crime in this context. They could start planning earlier or commit to action later.
Well the order wasn't lawful, Congress hadn't countenanced a war.
Unfortunately the military all turned their backs on their oath to uphold the constitution (Again! Deploying against USA cities and kidnapping foreign heads-of-state at the whim of Trump is also not lawful). At that point it is hard to see how your democracy can be resurrected.
<< At that point it is hard to see how your democracy can be resurrected.
Oddly, here I am more optimistic. The antipathy towards Tman rivals Musk and few other public personas and that, previously somewhat small crowd, appears to be growing. Historically speaking, I would say it is unlikely based on what happened ( or did not happen ) with Bush, but may actually unite enough people to punish its executive power center. I will admit that going into this year, this was not even close to my anything close to my bingo card. Then again, neither was Greenland, import war with Canada, Iran war and now open talk about banning diesel exports.
I don't want to get philosophical on you, but what is consent, exactly. For example, in US, most would claim the current administration has consent of the populace to govern and yet, somehow, priorities of said populations are nowhere close to being met suggesting that either the population's consent is either imaginary, temporary, conditional, a phrase or an more interesting mix of those. And amusingly, we do not even have to search hard for it, because this administration has a funny habit of saying quiet part out lout ( https://www.globalsecurity.org/military/library/news/2025/12... - "We don’t do foreign policy by poll, and our job is to carry out the directives of the President.").
On the other hand, we have Saudi Arabia, which, clearly, is not exactly a democracy. It may not be a dictatorship per se since it is a monarchy, but that monarchy rule is absolute ( so it is really a step away from NK ). And you may think they don't need population's consent, but you would be surprised how much consent you need when the population is used to 'harsh but oil is flowing so we get something out of it' trade off.
In other words, I am not saying you are wrong, but you may want to re-consider your priors.
Great point. Less deadly but just as incompetent was the utter failure to refill the Strategic Petroleum Reserve when gas was cheap for a year leading up to this war of choice; a war that was certain to massively disrupt global energy markets. Energy disruptions had been an obvious consequence to every expert and non-expert who had looked at attacking Iran for decades. Yet the people who chose to start the war managed to be caught completely off guard by that.
Just like clowns in charge were caught off guard by the fact that Iran has a drone program, resulting in the loss of dozens of air frames and much of our strategic munition stockpiles. Iran literally jump started Russia's drone program in a high profile way, and yet trump/hegseth somehow completely failed to plan for how to counter those drones and had to resort to lobbing scarce multi-million interceptor missiles at them and watching our bases take heavy damage.
These are just some of the failures we know about. The administration has been extremely tight lipped about these failures and there are surely many more that aren't common knowledge. Who knows how many hundreds of other children have been needlessly killed outside of the school strike.
from the perspective of an oil company when the price of oil goes up they make more money generally even if they sell less volume, it is in their interest that oil does not become to abundant that the price falls it is in their interest even if the economy suffers since oil is essential to many there will not be other option but to fill gas to go to work and they want to make the most money then even if eventually other energy alternatives start being considered since the executive team things about the bonus this year not next year or they year after that THIS year and what can be done THIS QUARTER
>If you're in a shooting war, the ability to select good military targets faster than your enemy, but not necessarily with zero error, is an advantage.
I mean, yes and no. Merely selecting the targets is not really an advantage in and of itself. I'm sure Iran could select hundreds of targets in the 48 States at the drop of a hat, but they don't have the air power to hit those targets so the air tasking cycle breaks down.
The air tasking cycle upon which modern NATO air campaigns are predicated and carried out is a cyclical 6-step process:
1. objectives not yet met/effects required
2. target development/selection to achieve those effects/meet those objectives
3. weaponeering, allocation, prioritization (can a hellfire launched from a drone do this or do we need something more specialized?)
4. actually write and issue the air tasking orders that cause air sorties to be flown
5. execution of said ATOs
6. assessment; GOTO 1
Normally this is a 72-hour cycle and your ops centre would be running 3+ of these cycles in overlapping fashion so that every 12-24 hours you have one loop reaching the top. AI appears to be in heavy use in the target development step, but there are diminishing returns for speed here. At some point there's just no added benefit for choosing more targets because the other loops are still running and you're potentially developing targets without knowledge of effects being delivered by the other ATO loops so you're now wasting effort. In other words it's not like a computer where more hertz is better; you might get better results by simply waiting for one of the other cycle's kinetic results that will better inform your own target development stage.
Except laundering our biases & discriminations, the unaccountability seems to be the purpose of a huge huge swarth of AI usage. To eliminate accountability. That sure seems like A Feature Not A Bug to the worlds worst people.
I'm not sure if it was any kind of official rule, but I worked at times in a JTAC capacity for troops in contact. I mention the last bit because it's very high stakes where seconds and minutes count a great deal more than when you do coordinated tactical strikes against strategic objectives like weapons caches. When I did that kind of work we had to have three eyes-on forms of contact. Often that was the calling troops, the observer (like an Air officer), and an air asset like a drone. They all had to independently describe the target, its orientation, and surrounding activity.
However you sourced your target is irrelevant to the activity at hand. When I first heard of this news I knew immediately someone skipped target verification or that it was simply no longer a policy of the DOD.
Famously, the last 'enemy' we killed in Afghanistan, alongside the 7 children and others surrounding him, was somebody who was working for a US aid organization. [1] A drone operator saw him putting bottles of water into his trunk, and somehow thought they were bombs, so they tracked him for 8 hours, then killed him and his family.
I actually ended all govt contracts I was working on after this and ended my clearance forever. This was my "I can't keep working for them" moment. One of the purest examples of the US operating on ISIS levels of morality that exists.
Killing someone for absurd reasons then lying about it because while knowing it's an absurdity/mistake risking it happening again seems almost worse than believing/buying into an extreme/insane worldview and killing someone for it.
This happened after I stopped working for the DOD, but I am 99% sure the program in question during this incident was something I worked on. It was effectively an anomaly detection system for identifying any weird behaviors detectable in WAMI (wide area motion imagery, which is high resolution, low framerate drone video data that covers a large geographic area). As an example from the unclassified sample project they asked us to do as part of the bidding process, our code flagged a car doing donuts in a parking lot (the sample data was just from a random US city on a random day because this was an unclassified open bidding process). The system was not and never was intended to be a "terrorist" detection system. The idea was always to flag potentially interesting events for a human analyst to follow up on. The disconnect it took for someone to blindly trust the program as a target acquisition system boggles my mind. Someone doing donuts in a parking lot should not be automatically targeted with hellfire missiles, yet in practice that is how they were using the tool. I still lose sleep over it.
Not sure about the downvotes, this is correct even if not pleasant to read (people need to be heroes of their little bullshit life story and all that).
It may be an afterthought for US citizens living in US, but its a deadly weapon pointing to 95% of the mankind and people wielding it are often shit to be polite. We the rest do care.
I mean, this site is full of people that develop the weapons used to commit atrocities and genocides (both physical and digital ones) so I am not surprised about the downvotes..
The answer is not to avoid engaging with a corrupt institution. US is a democracy: vote, be active politically, go into organizations that need change.
So you are suggesting that you can change the US military industrial complex by.. working for the NSA or Lockheed Martin ?
Also it is quite questionable how much of a democracy the US is nowadays.
Gerrymandering aside, the populace does not get to choose the presidential candidates. And the whole two party thing ? Or the fact that to have a chance you need a super PAC with hundreds of millions ?
Yes, that's what I'm suggesting. If you feel that the problem is higher up or lower down in the institutional hierarchy, then go higher up or lower down. That's the civil approach, as I understand it.
I'm probably too focused on the Afghanistan war but I don't think that voting differently would have extended or shortened the war.
Both republicans and democrats wanted the war in Afghanistan back when 9/11 happened and they needed to get revenge on the scapegoat. Meanwhile the end of the war was initiated by Trump, then slowly concluded by Joe Biden.
So both the start and the end of the war had bipartisan support. Nothing against democracy but war doesn't follow the election cycle. It might start in one though.
> Your work will be used against innocent civilians in far away lands.
For now. I think this administration will absolutely have no compunction agains using this against innocent civilians in THIS land as soon as they feel the need to to retain power.
that's why we outsource to other countries our software development, so you don't get blood on your hands violating the... essence of the constitution. i'm missing a word here.
hiring foreign data scientists and web developers.
Whenever you introduce (or grow) the presence of computer automation you need to consider the human / social interactions. You are responsible for the whole system, not just your software.
It is well understood that human diligence falls dramatically in the face of automation that is perceived to be highly reliable.
There is a tendency for software people to say “it has a 5% error rate, so make sure to carefully check every time”. Even though we know that humans simply won’t check carefully the results of a 95% accurate system.
> There is a tendency for software people to say “it has a 5% error rate, so make sure to carefully check every time”. Even though we know that humans simply won’t check carefully the results of a 95% accurate system.
This should really be framed differently during training. "Assume you have 100 neighbors in your area/house. We expect you to kill 5 innocent neighbors each time you use the system, and don't actually think for yourself. (After using it once every day for two weeks, you will have killed half of your neighbors.)"
Suddenly 5% doesn't sound that small/95% not as good to the laymen.
IMO you should increase the error rate synthetically, adding tests. So you get 10 targets to review, but 4 of them are known to be bad and the system is both checking the user, and also increasing the perceived error rate to encourage diligence.
Brilliant! This is the logic behind the birthday problem where you only need 45 people present to have greater than 90% chance that 2 people share a birthday.
Even rare events become very likely when you repeat it. A 5% wrong chance means half are dead after 10 tries
> Even rare events become very likely when you repeat it.
True, but in my opinion 5 innocent dead out of hundred is not remotely small. For any serious automated decision anything larger than per million is completely unacceptable. (For reference, among Germans the amount of Jews killed by Nationalsocialists, is "only" 2.3‰ [160k vs. 69M] I don't think anyone considers that to be small.)
Btw., after using it for a year every day, the US population would be 2.52 people. And I think the US military plans to use their expensive systems for much more than a year.
A computer can never be held accountable, therefore a computer must never make a management decision.
We have CEOs designing AI systems to replace them (or at least their workload). The DoD has done nothing to provide accountability for this war crime. It's a scapegoat to move on from, not a problem or justice.
The DoW rename (which Congress never approved) happened in 2025. The Department of Defense has had some major lapses in judgement for a long time before that.
This seems like a meaningless distinction to me. 'Overreliance on AI' implies the humans relied on the AI too much. You aren't even disagreeing with the article.
It didn't matte who/what looked at the maps - the maps were at least 11 years old and showed what was, at the time, a barracks. Nobody bothered to check for newer maps or verify it was still a barracks.
No, that's not what the poster wrote. See this part:
> "AI" doesn't really seem like the culprit -it's a scapegoat.
So the analysis was that AI has been a scapegoat. That's correct.
The implication is that it was indeed HUMANS who made the decision,
and now the US government tries to shift away the blames from
their decision onto "AI made a mistake, not we did". It is just not
logical to assume this was a mistake in the age of mass surveillance
and sniffing on everyone at all times.
GP says AI is not the culprit, but that actually agrees with the article + headline - which also blame the humans. Nobody in the article is scapegoating AI at all.
If you are both referring to some other claims by US gov made elsewhere (none are in the article), it might be useful to cite them.
Not if there was a clear and present danger. Note: I don't believe there was one (maybe someone convinced Trump there was); this seems to be more a "Shock & Awe" mission for political change a'la New American Century and has worked out similarly so far.
Something that I feel like I should have realized earlier but didn't until last year was how unbelievably lazy everyone involved is.
I knew they were stupid, I knew they had positions that I would consider racist, I knew that they were corrupt, but I guess I had assumed that they had worked for their corruption.
But no, you have RFK Jr.'s HHS lazily having ChatGPT generating anti-vax literature with broken links, you have Trump doing a blanket pardon of literally everyone involved with January 6 [1] apparently because he was too lazy to do any diligence. It's borderline farcical.
It is farcical. And then remember what a bunch of clowns the NSDAP's leaders were; the ranting Hitler, the esoteric nonsense of Himmler and Hess, Goering's makeup and fake uniforms, and then contrast that with their actions and impact on history. A clown-show widely mocked during their time, but look at the catastrophic impact they had: just because they're clowns doesn't mean they're not dangerous.
It's possible that it is, but if someone tried to prosecute it, state secrets privilege would like be invoked, as it has been in similar cases, so prosecutors don't even try.
I always said that AI is dangerous not because it's superintelligence will threaten humanity, but because some dumb/lazy people will overestimate it's abilities and use it where they should not.
This is just one more example of it. I'm waiting for even more disastrous ones.
All the AI leaders running around demanding everyone care about hypothetical "existential risk" and killing all of humanity are suspiciously silent on its active role in the emerging world war.
Anthropic’s leader objected so strenuously that their entire company was blacklisted from doing any business with the government, and blacklisted from any contractors working for the government from using Anthropic there either.
OpenAI’s agreement also included some restrictions, and they put out a press release about the restrictions.
So at least those AI leaders aren’t silent about AI’s military use.
That has long been my fear as well. I regularly encounter people who have given up all pretense of critical thought in favour of trusting some buggy computer system. And this started long before "AI" becoming prevalent.
Clearly the culprit is the person missing supervisor duties. However, AI, or rather how we use AI may well have contributed to the disaster.
Lack of oversight, over reliance and insufficient critical review lead to poor outcomes in any decision making process. Problem is, with AI more people ill-prepared for decision-making make decisions.
I am convinced that this is precisely how AI will cause more and more harm and much earlier than we can imagine.
I totally agree, but the US has done this for a long, long time. Elsewhere in this thread we were talking about the My Lai massacre, where the people involved also escaped punishment.
Correct. I wanted to reference My Lai as well. This is somewhat more “who the fuck cares” than Vietnam in my perception, when you had large and strong protest movements that are nowhere to be seen in that form or cultural relevance today.
This is NOT a generalization on all Americans by the way, that are in many cases lovely people that would agree with the above.
I also like what Bourdain said about Henry Kissinger and South East Asia, edit: specifically Cambodia.
I was baffled when the absolute heinous bastard finally fucking died to see how many people were paying their respect to basically Satan incarnated.
Here’s Bourdain:
Once you've been to Cambodia, you'll never stop wanting to beat Henry Kissinger to death with your bare hands. You will never again be able to open a newspaper and read about that treacherous, prevaricating, murderous scumbag sitting down for a nice chat with Charlie Rose or attending some black-tie affair for a new glossy magazine without choking. Witness what Henry did in Cambodia - the fruits of his genius for statesman- ship - and you will never understand why he's not sitting in the dock at The Hague next to Miloševic. While Henry continues to nibble nori rolls and remaki at A-list parties, Cambodia, the neutral nation he secretly and illegally bombed, invaded, undermined, and then threw to the dogs, is still trying to raise itself up
on its one remaining leg
There are so many crimes the US has committed that it still has not accounted for. On protests, I think there were a lot during the Biden era but the government and Zionist entities cracked down hard on them, especially the university protests. That obviously happened to an even more extreme degree during Vietnam (like the Kent State shootings), but the protests were even more intense then due to the draft. I think we'd see something similar if Americans started getting drafted to fight in Iran or anywhere else.
I feel like this actually highlights an undocumented feature of AI that institutions are taking advantage of: the ability to go "The AI fucked up. It happens. What're you gonna do?" I don't see anyone being held responsible and I don't see any plan to prevent this in the future.
I don't even understand why people are starting from an assumption of good faith like this (and the previous) administration haven't spent years helping the IDF destroy every civilian structure in Gaza taller than a tent. I promise you they don't think Iranian civilians are any more human than Palestinian ones.
I'm not anti-war, nor do I find all military intervention immoral, and I even support countries attacking back to disable military targets for good reason.
However, I'm absolutely against mistakes that kill children, and I hope that people are held responsible. This is not OK.
You don't see a push to use ai unconditionally and without verification to prove how much it's useful.
If the problem are human behaviours around ai, that is: eagerness to see the results (both military and Palantir), greediness to make money (Palantir), then the ai becomes the root of the problem.
> The Minab site, which was cataloged as an Islamic Revolutionary Guard Corps facility due to outdated data, was fed into Maven with other candidates and came out as a recommended day-one target. Target-list work that once took hours was condensed into minutes.
I feel like we're optimizing the wrong metric here.
Even ignoring the risk of civilian casualties (which to be clear should be weighed heavily) it’s just bad ROI. Like each missile probably costs more than the extra targeting work they should have done.
Well you see you don’t understand. We’re America, the biggest and best military like no one’s ever seen.
We have the largest stocks of missiles ever. We can build them faster than we use them and cheaper than anyone else. So you’re a liar saying it would have saved money doing it any other way
If ROI was even remotely a consideration the US military wouldn't be using multi-million-dollar missiles to blow up fishing skiffs.
If ROI was even remotely a consideration, the US military, the world's most complex logistics organization, would not be currently run by a man who couldn't even run a small (SMALL) veterans non-profit, and who spent most of his early months in the job issuing edicts about things like grooming standards and pissing off or forcing out people the US military has probably spent millions of dollars training into leadership positions.
You're assuming the goal is "kill the most Bad People (tm) for the lowest munition cost" where Bad People are who you and I presume are the Bad People.
It's not about cost. It's about doing what a fascist man-child wants done, as fast as possible, or hamberders are getting thrown, and apparently nobody in the US military has a spine.
It's about punishing and terrorizing a civilian population for something they have no control or say over.
Israel found that their AI system for figuring out "who is a Palestinian terrorist" - "Lavender" - which is a total black box - was terrible at actually IDing terrorists: https://www.972mag.com/lavender-ai-israeli-army-gaza/
So they "optimized the ROI" not by improving the black box by switching from multi-million-dollar smart bombs (which would level the supposed terrorist's home plus severely damage everything within a certain radius, but hey, omlettes, eggs, etc) to dumb bombs. Not much better than WW2/Vietnam era tech. Which now means they aim for a particular house, hit something within a couple of blocks and shout "huzzah!"
The goal isn't actually eliminating Palestinian terrorists. It's terrifying and slaughtering Palestinians.
Israel also "optimized ROI" by having snipers execute Palestinian children. The goal isn't killing terrorists. It's systematic genocide to be able to then claim the land there's nobody left to claim. https://www.theguardian.com/world/2024/apr/02/gaza-palestini...
If anyone thinks differently, look at the history of thousands of "special orders" from the military unit that "administers" Gaza and the West Bank. Israeli human rights groups have tried to catalog them all and been unable to because of the sheer number.
Just to give you an example of how stupid the special orders have been: some banned growing of flowers. To give you an example of how thorough the control is: you cannot attend school, operate a tractor, plant a crop, drill a well, or have more than a handful of people congregate without the military authority's authorization.
Is that really the case? Stories like this always have both sides muddying waters.
The terrorists are well known to use children and the rest of the public as hostage meatshields, and the US military is always looking for something/someone else to blame for the dirty work they have to do.
The broader public is so polarized by performative politics and botspam that they eat these stories up, but nothing I'm saying is secret. It's been this way since forever. Slow news today?
What fucking terrorists? We're talking about a country bombing another country's school. A country with a population of 93 million. Are you trying to insinuate that all 93 million Iranians are terrorists?
Maybe you meant the American terrorists?
> Stories like this always have all sides muddying waters.
All sides agree that the US bombed a school full of children, actually.
> Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that
This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure.
Chatbots in general can be quite opaque in how they operate, the biggest issue with agent products is communicating it's capabilities and limitations to users. If you have a bank app it's pretty clear whether or not it can send wire transfers, but a if that's a chatbot, users will have to think to ask it (and even then it could hallucinate)
No, this is what happens when software developers don't understand how users actually use their products.
This was a totally expected outcome, similar to how CV cameras arrest the wrong minorities in neighboring states 100s of miles away because it told the officer it was them; or how health insurance companies use ML algorithms to deny Americans care.
It's going to happen in other sectors of life and it will continue killing innocent civilians.
No, I'm saying health insurance companies routinely deny American civilians life saving care because they'd make more money and routinely use similar software that also allowed 200 school age girls to be murdered. Recalling that a CEO was murdered and the person excused is being hailed as a hero over just this. Think people care about this a lot actually.
For better or worse, health care providers at least in the US are for profit businesses. There's been plenty of arguments made to change that, but as it stands we shouldn't be surprised when a for profit business decides what service they provide and to whom they provide it.
Likewise, we shouldn't be surprised when someone who puts profits over other's lives gets ventillated by one of those lives in downtown New York. And what do you know, nobody was.
Actually, no. Murder is the deliberate killing of a person. If the person in your example believed the gun wouldn't go off, then it isn't murder. Negligent as hell, yes. Manslaughter, certainly. But not murder, because it doesn't meet the definition.
In my example, it would be murder, because just like here, nobody believes that the murderer believed the silly story about "I thought the murder weapon magically would refuse to murder".
The entire point of the post is that it isn't credible that the person actually thought their killing machine couldn't kill: pointing a gun at someone with the intention of killing, and pulling the trigger, counts as murder, even if you thought the person would magically survive if they were innocent.
You're free to try to make a case otherwise, but it'll be an uphill battle against reason.
Come on, they understand it perfectly well. They blew up a school! Of course they’ll happily blame the AI when journos nudge nudge wink wink suggest that they’ll take that as a perfectly good explanation.
The real headline is “Americans blow up school, blame AI”
The pentagon says it was Palantir software’s fault, Palantir says it was the fault of bad data input to their software. Where’s the accountability by both organizations? How did we get to a world where military decisions leading to innocent deaths are responded to as if it was just another miscommunication implementing a B2B SaaS platform.
Small children often develop coping mechanisms for their lack of agency. It may be part of these people’s worldview since early childhood that anyone who opposes them should magically die.
you can also develop pro-social coping mechanisms like open-mindedly talking to someone who opposes you and trying to understand their worldview because you trust that it might expand your own
but, yeah, I doubt pro-social traits are common at the top of any hyperstrict hierarchical organization like the USAF
The new world is complex and AI technology has outpaced the regulation cycle, so it's inevitable that we come to a place where you don't have accountability and can never have it because 0) the courts can't handle the case, they don't have the frameworks for it, so it takes longer time 1) you can overload the courts with slop, 2) by the time there is a decision the world has moved on and technology has moved on
I disagree with this framing. Nothing in accountability has really changed other than our willingness to enforce it.
Replace "AI" with any other tool or software in your scenario, and the outcome is the same. The same people and organizations are accountable, it's just that we currently live in a world without accountability in the highest places where it's most needed, and that's unrelated to AI.
You disagree that there is no difference between the amount of material you can generate today compared to 10 years ago?
Or with that the EU AI Act has been redundant and postponed in 2023 because it was drafted on ResNets, and then redundant and postponed in 2025 because it was drafted on ChatGPT 3.5, and then this year probably again because it didn't have agents included?
Or with that other countries that getting their companies hacked by AI models are ignoring any kind of prosecution of said companies and delegating this to the US?
You might be interested in Dan Davies' book The Unaccountability Machine[1].
It talks a fair bit about 'accountability sinks', which are exactly these systems that (intentionally or not) essentially serve as a way to unlink responsibility from any given person and allow 'the system' to be blamed instead.
Military accountability, and government accountability in general, has been and will always be a problem. As long as those in charge make and enforce the rules they will always look out for themselves first.
Well in theory the military is deferential to a political class that is elected by and thus accountable to the people. But that is so obviously no longer the case that politicians who pretend it is are perversely less popular than ones who are openly minoritarian.
The logical response to such a self-conscious rejection of democracy is unironically political education and beginning revolutionary political organizing in your neighborhood. But I rarely bother making that point here; HN is probably the least revolutionary-minded grouping in the world outside of tech billionaires themselves.
Do you expect any military to be held accountable legally and/or financially when they make a bad decision like this? It'd be great if they were, but I'd never expect it.
I don't think that's the example you want. Left to their own devices the Germans would not have held themselves accountable. As we speak the most popular party in Germany is a holocaust revisionist party.
Yes, though if you are referring to the Nuremberg Trials its not a great example. A huge number of Germans were aware of, complicit in, or directly involved in genocide...the trials had 24 defendants.
There was actual law enforcement in the Soviet army.
Hence, there was no genocide of the German people (which would be a completely justified and understandable action from any Soviet soldier's point of view).
408 comments
[ 0.24 ms ] story [ 107 ms ] threadFrom TFA.
Sometimes, the difference between a successful strike on a lawful combatant and disaster is one analyst pointing out that the building the enemy is hiding in is next to a square that’s used for a market on Tuesdays and wouldn’t you know it, it’s Tuesday. That’s not something the person “pulling the trigger,” can always be expected to know, which is why every ops center needs to be designed to elevate those voices, when the time comes
I cannot believe this has come to pass.
> The Pentagon chief last year slashed offices that didn’t contribute to his goal of “lethality,” including the group that assists in limiting risk to civilians, known as the Civilian Protection Center of Excellence. Around 200 employees who worked on the issue, including at that office, have been reduced by about 90 percent, according to two current and former officials and a person familiar with the effort. The team that handles civilian casualties at Central Command, which oversees the Middle East, has dropped from 10 to one.
Negligent homicide in pursuit of their DEI goals, basically.
And I hedged that. Anything short of 100% responsibility with humans is a failure. I get investigating how it occurred, but that needs be very careful to separate ought from is.
(setting aside when civilians are purposefully harmed which is a conversation about morals rather than the capability to make correct decisions)
What I am getting at is, has AI really made the problem worse or has the mechanism just changed?
When it was just called a "database query" or "algorithmic estimation", it was more implied that the actual decision is left to be made and that no analysis had actually been done yet. But when AI preemptively says "I've analyzed every source of data and come to the conclusion that this ship is carrying nuclear bomb material"; it really sounds urgent doesn't it? It probably doubled down when asked about it.
https://edition.cnn.com/2026/09/18/politics/us-military-ai-f...
> Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. Centcom’s group shrank from 10 people to one.
https://www.propublica.org/article/hegseth-civilian-harm-mit...
https://gizmodo.com/almost-started-a-war-us-military-nearly-...
The bigger story there is, like here, treating AI as a 'super intelligence' instead of a token prediction algorithm which is increasingly accurate, but still prone to hallucinations. It's like how self driving vehicles can drive with human to superhuman efficiency, yet then have failure modes like driving head first into a concrete divider. This isn't the sort of stuff you want touching anything critical without significant human oversight. Granted, at least in the case you're referencing that oversight did indeed exist, but it's not clear whether that was good fortune or systemic.
AI labs don't stand to benefit from the tech being understood as dangerous for its lack of intelligence. They need it to be understood as dangerous like an intelligent alien civilization.
https://gizmodo.com/trump-says-hes-changing-artificial-intel...
France gave them the boot, others can too
Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.
The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.
The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.
...
> Hegseth and the commander in charge of the Iran operation, Adm. Brad Cooper, were questioned about the strike before Congress. Hegseth said the U.S. does not target civilians and would investigate
So the administration does not intentionally target civilians, they have just stopped caring about who a civilian is.
https://en.wikipedia.org/wiki/Military-age_male
You might be confusing the approach for estimating combatant casualties after the fact. There were arguments made regarding the likelihood of military age casualties to actually have been combatants. That is not relevant. As far as who is considered a combatant during combat operations Israel follows international law.
EDIT: there is a lot of complexity here because Article 4 applies to soldiers (that e.g. fight in uniform) and not to organizations like Hamas and the PIJ that are considered "unlawful combatants" by Israel and fight in civilian clothing. But simply being a male of military age does not make one a combatant, lawful or legal, by Israel.
Basically they stopped caring those woke UN rules about war crimes.
FTFY. IMO I don't think they directly stopped caring about bombing civilians but have stopped caring about doing their jobs. And this staggering tragedy is the outcome.
I think with the atmosphere this current administration has created, questioning things is likely frowned upon. Better to just follow orders and if shit hits the fan, put on a circus act and move on to the next tragic disaster in the making. Someone else can always take the blame.
But don't worry, the other P AI system will ensure that the first one doesn't do mistakes...
UnitedHealth uses AI model with 90% error rate to deny care, lawsuit alleges - https://news.ycombinator.com/item?id=38299921 - November 2023 (58 comments)
https://litigationtracker.law.georgetown.edu/wp-content/uplo...
the situation would have to be very particular for that not to be the case.
The situation is very particular. When the system makes an error, I make a bigger profit. If the system made fewer errors, it would approve more things that cost me money.
Why do you think I'm motivated to make fewer errors under that situation?
https://en.wikipedia.org/wiki/Type_I_and_type_II_errors
Sure if you're flipping a coin.
If you're calculating a number from 1 to 100 then depending on distribution a calculation that's wrong 90% of the time is probably useless.
member the three words "delay, deny, depose"? Insurances - and not just in the US, Germany has similarly bad stuff going on - aren't making a profit if they just blindly accept claims. It is much more profitable to deny with a vague "AI" system (or a blanket automated deny) first, and only have a first look with an actual human agent at it when the customer complains or files a lawsuit.
For that reason error rates above 50% don't make any sense. Building a system, that is reliably wrong, is as hard as building one that is reliable correct, because the systems are the same, just with the output inverted.
The worst you can get with a system is 50%, at which point you can just flip a coin, because that is just pure random. Any deviation from that is going to be an improvement.
I'm not sure why the "people make mistakes too" crowd can't seem to grasp this concept.
But that's a US-specific thing. With humans you at least have this theoretical liability-- that there is someone who could be punished.
and so is the Trump administration?
It's the difference between a sword and a nuclear missile even though both are "technology"
The pattern matches various other acts of senseless violence against innocents in South America.
The message isn't "We did this accidentally. We should understand how it happened, so we can make sure it doesn't happen again.".
It's "We do this to make the point that we can, no one can stop us, and if we're challenged we're going to lie about it."
They blamed excel for the bad choice. Excel.
I mean sure, you can always blame the tool, i can blame my cars rear parking sensors when i hit someone when backing up during parking, but come on... at some point, a human is responsible. Someone, some person, decided and greenlit the decision to bomb those places and should now be in jail for that.
I mean, what's next? A building collapses, because the load bearing sections are too weak... do we blame CAD tools now?
So AI’s output is not so different than a roll of dice? Definitely the people who decided to put in such an incapable and unreliable tool should be held accountable. No dispute in that.
>The decision was indeed made by AI and humans failed in their duty to fucking verify it.
The decision to leave the decision to AI was made by humans. It's on the moronic humans for not understanding the limitations and flaws of the software they are using and believing the marketing material without constantly verifying their results.
The decision, to make the decision to leave the decision…
So much mental gymnastics.
FWIW, forcing people to go along with transparent lies is a popular social manoeuvre. It signals both the power of a group and the power of the person leading the lie.
Although it is a very frustrating tendency, because honesty and realism tends to be better for the people who have the numbers. It is hard to organise real efforts around lies.
Before anyone does this please let me know beforehand so I can invest in paperclip factories…
The US government just tries to shift blame for atrocities. This also, by the way, coincides with the US government trying to abolish a world court investigating war crimes. Hopefully we'll have an objective world court one day, so ALL war criminals get prosecuted, at all times.
> Some US intelligence analysts took notice, the officials said. Bloomberg reported in June that one analyst detected changes to the site as early as 2019, and logged remarks about the modifications in a system that wasn’t connected to the primary military intelligence database, which informs targeting
At the time a news outlet put out commercially purchased images of the target site before and after -- even a layman could see that it had a children's playground next to the building, and that it was separated from the compound.
https://www.channel4.com/news/factcheck/factcheck-iran-schoo... has the info from around the time, but doesn't show the detailed images in the news report (video) I seem to remember seeing.
I think it was maybe some earlier coverage from NYT, they published this retrospectively - https://www.nytimes.com/video/world/middleeast/1000000107503... (which is also on YouTube, https://www.youtube.com/watch?v=hDcim48W4Io). It shows that even a cursory analysis - without the benefit of enhanced military grade satellite/flyover imagery - indicates the building targeted with pin-point accuracy is a school (eg https://youtu.be/hDcim48W4Io?t=98).
In short, this seems entirely avoidable. I don't think you need any connected system, you just need an image of the target and eyes.
If they trained a model to find targets and didn't train it to avoid civilian targets, and couldn't be bothered to have anyone check them, that's still a war crime. The analyst could probably see the kids playing there with the imaging tech they have available.
This war was unlawfully started by the military at Trump's command, knowing it was not a lawful order; just as they knew firing on shipwrecked civilian sailors was murder. Seemingly it was started to win money betting on fuel prices.
Now if you ACTUALLY used AI, as opposed to what happened here (Trump stooge getting request for 1000 targets and doing SELECT * FROM targets LIMIT 1000, except getting AI to write the query) could could drive the odds of this happening much closer to zero.
[1]https://www.bbc.com/news/articles/c5yqqyly9n0o
-------------------
Maven is an untested tool for rapidly selecting targets under time pressure during a war. It's reason for existing is speed. If you're in a shooting war, the ability to select good military targets faster than your enemy, but not necessarily with zero error, is an advantage.
Planning a first strike is a different situation. Time pressure should not be a significant factor. Humans should have been doing their due diligence on every single target. If an untested tool for rapid mid-battle target selection was relied upon completely because of a hasty executive order, then the responsibility falls entirely on the executive.
Unfortunately the military all turned their backs on their oath to uphold the constitution (Again! Deploying against USA cities and kidnapping foreign heads-of-state at the whim of Trump is also not lawful). At that point it is hard to see how your democracy can be resurrected.
Oddly, here I am more optimistic. The antipathy towards Tman rivals Musk and few other public personas and that, previously somewhat small crowd, appears to be growing. Historically speaking, I would say it is unlikely based on what happened ( or did not happen ) with Bush, but may actually unite enough people to punish its executive power center. I will admit that going into this year, this was not even close to my anything close to my bingo card. Then again, neither was Greenland, import war with Canada, Iran war and now open talk about banning diesel exports.
That would make me optimistic as well if dictatorships governed with the consent of the populace. They do not.
On the other hand, we have Saudi Arabia, which, clearly, is not exactly a democracy. It may not be a dictatorship per se since it is a monarchy, but that monarchy rule is absolute ( so it is really a step away from NK ). And you may think they don't need population's consent, but you would be surprised how much consent you need when the population is used to 'harsh but oil is flowing so we get something out of it' trade off.
In other words, I am not saying you are wrong, but you may want to re-consider your priors.
Just like clowns in charge were caught off guard by the fact that Iran has a drone program, resulting in the loss of dozens of air frames and much of our strategic munition stockpiles. Iran literally jump started Russia's drone program in a high profile way, and yet trump/hegseth somehow completely failed to plan for how to counter those drones and had to resort to lobbing scarce multi-million interceptor missiles at them and watching our bases take heavy damage.
These are just some of the failures we know about. The administration has been extremely tight lipped about these failures and there are surely many more that aren't common knowledge. Who knows how many hundreds of other children have been needlessly killed outside of the school strike.
I mean, yes and no. Merely selecting the targets is not really an advantage in and of itself. I'm sure Iran could select hundreds of targets in the 48 States at the drop of a hat, but they don't have the air power to hit those targets so the air tasking cycle breaks down.
The air tasking cycle upon which modern NATO air campaigns are predicated and carried out is a cyclical 6-step process:
1. objectives not yet met/effects required
2. target development/selection to achieve those effects/meet those objectives
3. weaponeering, allocation, prioritization (can a hellfire launched from a drone do this or do we need something more specialized?)
4. actually write and issue the air tasking orders that cause air sorties to be flown
5. execution of said ATOs
6. assessment; GOTO 1
Normally this is a 72-hour cycle and your ops centre would be running 3+ of these cycles in overlapping fashion so that every 12-24 hours you have one loop reaching the top. AI appears to be in heavy use in the target development step, but there are diminishing returns for speed here. At some point there's just no added benefit for choosing more targets because the other loops are still running and you're potentially developing targets without knowledge of effects being delivered by the other ATO loops so you're now wasting effort. In other words it's not like a computer where more hertz is better; you might get better results by simply waiting for one of the other cycle's kinetic results that will better inform your own target development stage.
Except laundering our biases & discriminations, the unaccountability seems to be the purpose of a huge huge swarth of AI usage. To eliminate accountability. That sure seems like A Feature Not A Bug to the worlds worst people.
However you sourced your target is irrelevant to the activity at hand. When I first heard of this news I knew immediately someone skipped target verification or that it was simply no longer a policy of the DOD.
[1] - https://www.bbc.com/news/world-us-canada-58604655
This is a good lesson for young people reading this. If you work for the US military, you are not working for Defence.
Your work will be used against innocent civilians in far away lands.
It may be an afterthought for US citizens living in US, but its a deadly weapon pointing to 95% of the mankind and people wielding it are often shit to be polite. We the rest do care.
Also it is quite questionable how much of a democracy the US is nowadays.
Gerrymandering aside, the populace does not get to choose the presidential candidates. And the whole two party thing ? Or the fact that to have a chance you need a super PAC with hundreds of millions ?
This seems like elitocracy..
Both republicans and democrats wanted the war in Afghanistan back when 9/11 happened and they needed to get revenge on the scapegoat. Meanwhile the end of the war was initiated by Trump, then slowly concluded by Joe Biden.
So both the start and the end of the war had bipartisan support. Nothing against democracy but war doesn't follow the election cycle. It might start in one though.
[0] https://en.wikipedia.org/wiki/Real_Genius
For now. I think this administration will absolutely have no compunction agains using this against innocent civilians in THIS land as soon as they feel the need to to retain power.
hiring foreign data scientists and web developers.
It is well understood that human diligence falls dramatically in the face of automation that is perceived to be highly reliable.
There is a tendency for software people to say “it has a 5% error rate, so make sure to carefully check every time”. Even though we know that humans simply won’t check carefully the results of a 95% accurate system.
This should really be framed differently during training. "Assume you have 100 neighbors in your area/house. We expect you to kill 5 innocent neighbors each time you use the system, and don't actually think for yourself. (After using it once every day for two weeks, you will have killed half of your neighbors.)"
Suddenly 5% doesn't sound that small/95% not as good to the laymen.
Even rare events become very likely when you repeat it. A 5% wrong chance means half are dead after 10 tries
True, but in my opinion 5 innocent dead out of hundred is not remotely small. For any serious automated decision anything larger than per million is completely unacceptable. (For reference, among Germans the amount of Jews killed by Nationalsocialists, is "only" 2.3‰ [160k vs. 69M] I don't think anyone considers that to be small.)
We have CEOs designing AI systems to replace them (or at least their workload). The DoD has done nothing to provide accountability for this war crime. It's a scapegoat to move on from, not a problem or justice.
glad to see you're alive and functioning
> "AI" doesn't really seem like the culprit -it's a scapegoat.
So the analysis was that AI has been a scapegoat. That's correct.
The implication is that it was indeed HUMANS who made the decision, and now the US government tries to shift away the blames from their decision onto "AI made a mistake, not we did". It is just not logical to assume this was a mistake in the age of mass surveillance and sniffing on everyone at all times.
GP says AI is not the culprit, but that actually agrees with the article + headline - which also blame the humans. Nobody in the article is scapegoating AI at all.
If you are both referring to some other claims by US gov made elsewhere (none are in the article), it might be useful to cite them.
It all makes sense. So what we had to blow up some savages to steal their oil. Whatever, you don’t expect Jeb to push his truck do you.
The Russians , who have a different relationship to race than we do , have treated Ukrainians like Brown people.
How barbaric. To treat “European Christians” in such a manner.
Russia bad.
Anyway with the price of gas never coming down I guess Jeb will need to buy a bike.
I knew they were stupid, I knew they had positions that I would consider racist, I knew that they were corrupt, but I guess I had assumed that they had worked for their corruption.
But no, you have RFK Jr.'s HHS lazily having ChatGPT generating anti-vax literature with broken links, you have Trump doing a blanket pardon of literally everyone involved with January 6 [1] apparently because he was too lazy to do any diligence. It's borderline farcical.
[1] https://www.axios.com/2025/01/22/trump-pardons-jan6-clemency
https://en.wikipedia.org/wiki/Killing_of_Alex_Pretti
This is just one more example of it. I'm waiting for even more disastrous ones.
Anthropic’s leader objected so strenuously that their entire company was blacklisted from doing any business with the government, and blacklisted from any contractors working for the government from using Anthropic there either.
OpenAI’s agreement also included some restrictions, and they put out a press release about the restrictions.
So at least those AI leaders aren’t silent about AI’s military use.
Lack of oversight, over reliance and insufficient critical review lead to poor outcomes in any decision making process. Problem is, with AI more people ill-prepared for decision-making make decisions.
I am convinced that this is precisely how AI will cause more and more harm and much earlier than we can imagine.
source?
I also like what Bourdain said about Henry Kissinger and South East Asia, edit: specifically Cambodia. I was baffled when the absolute heinous bastard finally fucking died to see how many people were paying their respect to basically Satan incarnated.
Here’s Bourdain: Once you've been to Cambodia, you'll never stop wanting to beat Henry Kissinger to death with your bare hands. You will never again be able to open a newspaper and read about that treacherous, prevaricating, murderous scumbag sitting down for a nice chat with Charlie Rose or attending some black-tie affair for a new glossy magazine without choking. Witness what Henry did in Cambodia - the fruits of his genius for statesman- ship - and you will never understand why he's not sitting in the dock at The Hague next to Miloševic. While Henry continues to nibble nori rolls and remaki at A-list parties, Cambodia, the neutral nation he secretly and illegally bombed, invaded, undermined, and then threw to the dogs, is still trying to raise itself up on its one remaining leg
...or it was just a XLS/Excel sheet sent around by someone by mail
However, I'm absolutely against mistakes that kill children, and I hope that people are held responsible. This is not OK.
If the problem are human behaviours around ai, that is: eagerness to see the results (both military and Palantir), greediness to make money (Palantir), then the ai becomes the root of the problem.
I feel like we're optimizing the wrong metric here.
We have the largest stocks of missiles ever. We can build them faster than we use them and cheaper than anyone else. So you’re a liar saying it would have saved money doing it any other way
/s
If ROI was even remotely a consideration, the US military, the world's most complex logistics organization, would not be currently run by a man who couldn't even run a small (SMALL) veterans non-profit, and who spent most of his early months in the job issuing edicts about things like grooming standards and pissing off or forcing out people the US military has probably spent millions of dollars training into leadership positions.
You're assuming the goal is "kill the most Bad People (tm) for the lowest munition cost" where Bad People are who you and I presume are the Bad People.
It's not about cost. It's about doing what a fascist man-child wants done, as fast as possible, or hamberders are getting thrown, and apparently nobody in the US military has a spine.
It's about punishing and terrorizing a civilian population for something they have no control or say over.
Israel found that their AI system for figuring out "who is a Palestinian terrorist" - "Lavender" - which is a total black box - was terrible at actually IDing terrorists: https://www.972mag.com/lavender-ai-israeli-army-gaza/
So they "optimized the ROI" not by improving the black box by switching from multi-million-dollar smart bombs (which would level the supposed terrorist's home plus severely damage everything within a certain radius, but hey, omlettes, eggs, etc) to dumb bombs. Not much better than WW2/Vietnam era tech. Which now means they aim for a particular house, hit something within a couple of blocks and shout "huzzah!"
The goal isn't actually eliminating Palestinian terrorists. It's terrifying and slaughtering Palestinians.
Israel also "optimized ROI" by having snipers execute Palestinian children. The goal isn't killing terrorists. It's systematic genocide to be able to then claim the land there's nobody left to claim. https://www.theguardian.com/world/2024/apr/02/gaza-palestini...
If anyone thinks differently, look at the history of thousands of "special orders" from the military unit that "administers" Gaza and the West Bank. Israeli human rights groups have tried to catalog them all and been unable to because of the sheer number.
Just to give you an example of how stupid the special orders have been: some banned growing of flowers. To give you an example of how thorough the control is: you cannot attend school, operate a tractor, plant a crop, drill a well, or have more than a handful of people congregate without the military authority's authorization.
https://www.propublica.org/article/hegseth-civilian-harm-mit...
The terrorists are well known to use children and the rest of the public as hostage meatshields, and the US military is always looking for something/someone else to blame for the dirty work they have to do.
The broader public is so polarized by performative politics and botspam that they eat these stories up, but nothing I'm saying is secret. It's been this way since forever. Slow news today?
Which makes the administration's near complete silence on the strike rather suspicious, no?
Yes it is really the case here.
What fucking terrorists? We're talking about a country bombing another country's school. A country with a population of 93 million. Are you trying to insinuate that all 93 million Iranians are terrorists?
Maybe you meant the American terrorists?
> Stories like this always have all sides muddying waters.
All sides agree that the US bombed a school full of children, actually.
This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure.
This was a totally expected outcome, similar to how CV cameras arrest the wrong minorities in neighboring states 100s of miles away because it told the officer it was them; or how health insurance companies use ML algorithms to deny Americans care.
It's going to happen in other sectors of life and it will continue killing innocent civilians.
https://www.newsweek.com/united-healthcare-ceo-shooting-ai-l...
https://www.ama-assn.org/press-center/ama-press-releases/phy...
https://www.palmbeachpost.com/story/news/healthcare/2026/03/...
Forgive my ignorance, I just don't get the reference and don't want to assume its anything front he pandemic response to United Healthcare.
Still murder.
You're free to try to make a case otherwise, but it'll be an uphill battle against reason.
The real headline is “Americans blow up school, blame AI”
you can also develop pro-social coping mechanisms like open-mindedly talking to someone who opposes you and trying to understand their worldview because you trust that it might expand your own
but, yeah, I doubt pro-social traits are common at the top of any hyperstrict hierarchical organization like the USAF
Really good time to do various crime.
Replace "AI" with any other tool or software in your scenario, and the outcome is the same. The same people and organizations are accountable, it's just that we currently live in a world without accountability in the highest places where it's most needed, and that's unrelated to AI.
Or with that the EU AI Act has been redundant and postponed in 2023 because it was drafted on ResNets, and then redundant and postponed in 2025 because it was drafted on ChatGPT 3.5, and then this year probably again because it didn't have agents included?
Or with that other countries that getting their companies hacked by AI models are ignoring any kind of prosecution of said companies and delegating this to the US?
It talks a fair bit about 'accountability sinks', which are exactly these systems that (intentionally or not) essentially serve as a way to unlink responsibility from any given person and allow 'the system' to be blamed instead.
[1] https://en.wikipedia.org/wiki/The_Unaccountability_Machine
The logical response to such a self-conscious rejection of democracy is unironically political education and beginning revolutionary political organizing in your neighborhood. But I rarely bother making that point here; HN is probably the least revolutionary-minded grouping in the world outside of tech billionaires themselves.
> That didn't happen.
> And if it did, it wasn't that bad.
> And if it was, it was AI's fault, not mine.
It's gonna say something like "they were all military-aged males". https://en.wikipedia.org/wiki/Military-age_male