The Lawfare Podcast · The Lawfare Institute

Lawfare Archive: Elliot Jones on the Importance and Current Limitations of AI Testing

March 15, 2026·39 min·3 clips

Elliot Jones reveals why testing AI is like Henry Ford inventing a car, helicopter, and submarine all at once.

This Lawfare Archive episode, introduced by intern Marissa Long with context about Anthropic's 2026 lawsuit against the Department of Defense, replays an August 2024 conversation between Kevin Frazier and Elliot Jones about the state of AI systems testing. Jones, a senior researcher at the Ada Lovelace Institute, explains that his team spent six months producing a comprehensive report on AI evaluation because no such guide existed despite rapid advances in AI capabilities. He defines key terms: evaluations broadly try to understand what a model can do and what impacts it has; benchmarks are standardized question sets that allow cross-model comparison; and audits are structured, standardized assessments that may extend beyond the model itself to governance practices. The central difficulty with testing foundation models, Jones argues, is their generality. Unlike a chest X-ray system with a defined purpose, large language models can be applied in finance, education, healthcare, and countless other domains. Developers themselves do not fully know how their systems will be used once released, and they cannot explain why the models produce specific outputs. Jones compares this to Henry Ford simultaneously inventing a car, helicopter, and submarine and then asking regulators to figure out the risks. On the regulatory landscape, Jones notes the EU is furthest ahead with the European AI Act, which mandates assessment of general-purpose systems for systemic risks including cyber warfare and systemic discrimination. The European AI Office is consulting on codes of practice expected around spring 2025. The UK AI Safety Institute operates on a voluntary basis, having recruited technical talent from companies like OpenAI and DeepMind to build independent evaluations. The US AI Safety Institute is still being established and collaborates closely with the UK. Jones identifies a collective action problem in voluntary testing regimes: companies that share their models risk having their flaws publicly exposed while competitors who do not participate face no scrutiny. He notes legitimate concerns around commercial sensitivity, since leaked model weights could devastate a company like OpenAI. When companies build their own benchmarks, they effectively have both the test and the answer sheet. The conversation addresses what happens after testing reveals problems. Options range from adjusting training data to address bias, adding safety filters for harmful prompts, monitoring outputs post-deployment, and in rare cases withholding models entirely. Jones cites OpenAI's decision not to release a voice cloning model capable of realistic audio generation as an example of responsible restraint, while acknowledging that commercial pressure to recoup billion-dollar training costs will increasingly push against such caution. On the question of audit washing, Jones emphasizes that current evaluations should be treated as indicative rather than definitive. Without post-market monitoring and incident reporting to connect pre-release test results to real-world outcomes, no set of evaluations should serve as a safety stamp of approval. He notes that even implementing an off-the-shelf benchmark required Anthropic engineers to spend months of work. The report was based on interviews with foundation model developers, third-party auditors, government officials, and academics, many of whom requested anonymity. Jones acknowledges some findings were already outdated by publication, given the pace of the field. He recommends that regulators be clear about which evaluations they require, noting that both companies and researchers have expressed desire for that guidance.

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from The Lawfare Podcast