RAW co-director Jessica Dorsey, together with Sarah Shoker (UC Berkeley Risk and Security Lab, formerly OpenAI’s Geopolitics Team Lead), has published a new piece in Opinio Juris: “Testing the Limits: AI-Enabled Targeting, Model Evaluation and International Humanitarian Law.”
The piece intervenes in the ongoing debate over AI companies’ self-imposed “red lines” for military use, arguing that this framing diverts attention from the systems already being built and deployed today: AI-enabled decision-support tools that help identify, prioritize, and validate targets. Dorsey and Shoker note that these tools are credited by the U.S. with generating roughly 1,000 targets a day during the Iran campaign, a tempo they compare to the “shock and awe” logic of earlier wars.
Their central claim is a methodological one: current LLM safety testing is largely self-regulated by frontier labs and tells us little about how these systems perform once embedded in time-constrained, hierarchical, high-pressure targeting environments. Benchmarks built for competitive comparison say nothing about over-reliance, automation bias, or the gradual erosion of operator judgment under operational tempo. Drawing on investigative work from Airwars and a wider group of collaborators including Zena Assaad, Elke Schwarz, Ingvild Bode, Damian Copeland, Neil Renic, Marta Bo, Luke Moffett, and Taylor Kate Woodcock, the authors argue that the proper object of evaluation is not the model in isolation but the sociotechnical system in which it operates: model, data, interface, operators, command structure, and tempo together.
This leads to a proposed reversal of the evidentiary burden: rather than waiting for evidence of harm after deployment, states should be required to demonstrate, before deployment, that these systems can support rather than undermine the human judgment IHL requires, with companies expected to supply the transparency needed to make that demonstration possible.
The piece is a valuable companion to RAW’s own work on the “beforemath” of algorithmic harm, and to the group’s continued engagement with the accountability gaps opened up by AI-assisted targeting.
Read the full piece at Opinio Juris.
