Your Undivided Attention · The Center for Humane Technology, Tristan Harris, Daniel Barcay and Aza Raskin

War is a Laboratory for AI with Paul Scharre

May 23, 2024·59 min·5 clips

In 1983, a Soviet officer ignored computers warning of a U.S. nuclear strike—would AI make the same call?

Paul Scharre, author of two books on autonomous weapons and former Pentagon policy architect, examines how AI is transforming warfare based on recent observations in Ukraine. He opens by recounting the 1983 Stanislav Petrov incident where a Soviet officer's human judgment prevented nuclear war by ignoring false computer alerts about incoming U.S. missiles. Scharre recently visited Ukraine, describing it as an unprecedented laboratory for AI weapons innovation where terminal guidance autonomy helps drones complete attacks despite jamming. The conversation establishes definitional boundaries: remotely piloted systems versus autonomous weapons that identify and engage targets independently after launch, analogous to the progression from driver assistance features to fully self-driving cars. Scharre explains military incentives include speed advantages in split-second scenarios and maintaining functionality when communications are jammed. He distinguishes weapon autonomy from broader military AI applications in logistics, maintenance, and personnel management where efficiency drives adoption. Two competing narratives emerge: precision advocates argue AI reduces emotionally-driven mistakes and war crimes by sleep-deprived soldiers, while critics fear normalizing violence when cold calculation replaces human moral restraint. Scharre validates both possibilities depending on context and who deploys the technology. He recounts a personal Afghanistan experience where his sniper team encountered a five-year-old shepherd girl scouting for Taliban fighters—legally targetable under law of war but morally wrong, illustrating how AI programmed for perfect legal compliance might make unethical decisions. The discussion examines philosophical urgency: Nick Bostrom's concept of AI as "philosophy on a deadline" captures how 18th-century institutions struggle articulating ethical distinctions as fast as technology demands them. Scharre identifies tasks suitable for automation (clear performance metrics like landing aircraft) versus judgment-dependent contexts requiring human understanding of ambiguous situations. He notes Google's Project Maven withdrawal forced Pentagon transparency, leading to AI ethics principles, though implementation challenges remain translating lofty principles into practical guidance. Scharre expresses comfort with U.S. military's ethical approach but worries about Chinese prioritization of political control over legal compliance and Russian chemical weapons use in Ukraine indicating disregard for war law constraints. The concept of "hyperwar" emerges—when competitive OODA loop acceleration creates superhuman decision speeds analogous to high-frequency trading flash crashes, but without circuit breakers or referees. This raises the specter of "mutually assured loss of control" where escalatory loops exceed human intervention capacity. The tone combines technical military expertise with philosophical reflection, using concrete examples like the shepherd girl and stock market crashes to ground abstract concepts. Scharre demonstrates insider knowledge while maintaining critical distance on autonomous weapons risks. The conversation remains measured and analytical, avoiding both technophobic panic and uncritical optimism. This episode suits audiences interested in military ethics, AI governance, international security, and technology policy. Listeners seeking practical understanding of how autonomous weapons actually function and evolve will appreciate the technical grounding. Those concerned about authoritarian applications or institutional adaptation to technological change will find substantive engagement with those specific challenges.

As heard by us

A careful look at how AI meets military judgment under the shadow of nuclear error.

This episode begins with Stanislav Petrov's 1983 nuclear false alarm and uses that event to frame a sharper question: whether an AI-based system would have made the same call.

Read the full review in PlayNext →

Why you'd press play

You want the real-world stakes of military AI, not buzzwords.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Your Undivided Attention