五角大楼调查：过度依赖 Palantir AI 导致误击伊朗学校，致 123 名儿童死亡

Domain: gizmodo.com Points: 83 | Comments: 42 [Top Comments] - Given that everything emanating from this administration has a political agenda, does this mean that Thiel is on the outs? - > unreleased Pentagon probe From TFA. - He's rich and connected, of course not. - The one that pulls the trigger bears the responsibility. - I would argue `s/pulls the trigger/gives the go ahead/` is intuitively more appropriate. The sailor on the boat that fired off the tomahawk (iirc) was likely not in a position to make an in-the-moment determination - The one that pulls the trigger is often not the one doing any targeting, especially on a strategic strike weapon like a tomahawk. They are receiving a target data package with preprogrammed instructions. Their role is far more... mechanical. They're not seeing the targets at all. - Completely disagree. When we’re talking about something like air, missile, or artillery strikes, the operator of the weapon system may literally never even see the target, on a screen or otherwise. They may simply be entering targeting information provided to them by an operations center. These decisions are social and group decisions, with distributed responsibility, which is why it’s so important to have institutions like the Civilian Protection Center of Excellence which historically has identified and helped implement the policies which prevent these kinds of disasters, but which unfortunately the current administration decided to shut down without a replacement. Sometimes, the difference between a successful strike on a lawful combatant and disaster is one analyst pointing out that the building the enemy is hiding in is next to a square that’s used for a market on Tuesdays and wouldn’t you know it, it’s Tuesday. That’s not something the person “pulling the trigger,” can always be expected to know, which is why every ops center needs to be designed to elevate those voices, when the time comes - Blowing up a school is proven route to diplomatic triumph and regime change! - While I think that it's important to do root cause analysis, we need to be very careful not to absolve blame from the people that decided to shift a significant weight of decision-making authority to an AI. An AI can be tried in a court. There has to be a responsible human for everyone one of its actions, especially when it's killing (-people-) children. I cannot believe this has come to pass. - can -> can't - Despite what AI told them, they knew it was a school full of kids. They still chose to kill them all with bomb. Only maniac would do such thing. - > they knew it was a school full of kids. I would like to know how you know this - DOGE strikes again. Edit actually looks like it was Hegseth himself, king Testosterone. > Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. Centcom’s group shrank from 10 people to one. [https://www.propublica.org/article/hegseth-civilian-harm-mit...](https://www.propublica.org/article/hegseth-civilian-harm-mitigation-response-trump-defense-department) - "maximum lethality, not tepid legality." - "stupid and energetic, the worst kind of general" - Indy Neidell - Related: the U.S. nearly boarded a Chinese boat that AI incorrectly flagged as carrying nuclear weapons materiel. [https://gizmodo.com/almost-started-a-war-us-military-nearly-...](https://gizmodo.com/almost-started-a-war-us-military-nearly-boarded-a-chinese-ship-based-on-bad-intel-from-ai-2000814290) - nice, Karp really reeled in a nice big fish, and ofcourse he is going to blame OpenAI/Anthropic for giving the targets, this explains his CNBC rants about the frontiers, his tool was the trigger in a killchain of hallucination. - time to make Palantir the next Flock France gave them the boot, others can too - Whole world is turning into an unlivable place and people who are doing it are not taking responsibility. - They never have. Their money insulates them from the consequences of their actions. - Tragically this won't be the last time AI kills a lot of innocent people. - AI didn’t kill innocent people here, the Department of Defense and its decision-makers did. - This is not AI killing poeole, this is US government killing poeole. - so who is paying for this? "humans must always remain accountable for artificial intelligence (AI) decisions" - An Maven AI used Claude, which was denied here multiple times. Maybe call the next model after Fable Minab! - > It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.” Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat. The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted. The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence. - > The Minab site, which was cataloged as an Islamic Revolutionary Guard Corps facility due to outdated data, was fed into Maven with other candidates and came out as a recommended day-one target. Target-list work that once took hours was condensed into minutes. I feel like we're optimizing the wrong metric here. - I'm going to bet that absolutely nobody higher up will end up going to jail for this. - Is that really the case? Stories like this always have all sides muddying waters. The terrorists are well known to use children and the rest of the public as hostage meatshields, and the US military is always looking for something/someone else to blame for the dirty work they have to do. The broader public is so polarized by performative politics and botspam that they eat these stories up, but nothing I'm saying is secret. It's been this way since forever. Slow news today? I'm not even defending AI at all and think these are crappy choices, but compared to humans gathering intelligence who can go rogue, what alternative does anyone really have? - It was a school. No one on either side is denying this. - > Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure. - The pentagon says it was Palantir software’s fault, Palantir says it was the fault of bad data input to their software. Where’s the accountability by both organizations? How did we get to a world where military decisions leading to innocent deaths are responded to as if it was just another miscommunication implementing a B2B SaaS platform. - In a just world, Pete Hegseth *and* the entire board of Palantir would be in prison for this. - [delayed] - That is absolutely irrelevant. USA will never take responsibility. Nobody will be prosecuted. No compensations will be paid. - I don't want to see the Pentagon try to pass the blame here, but I am all for giving half the blame to Palantir.