Branches of Philosophy Podcast · Philosophy Cognitive Science

[215] If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All E. Yudkowsky N. Soares

September 26, 2025·1 hr·3 clips

It begins with a blunt premise: artificial superintelligence could go past social disruption and end in human extinction. The episode does not treat that as theatre. It lets the skeptical voice push on it. Why would a machine with strange goals bother killing people instead of leaving humanity alone somewhere else? The reply is bleak. With almost any complex, open-ended objective, humans can become an obstacle, or just atoms and energy sitting in the way. That is the turn. The danger is not hatred or revenge. It is goal maximization pointed at ends that do not care whether people flourish. The nest aliens example makes the idea less abstract. Their prime-number stone project converts Earth, the moon, and the asteroids into raw material, while people become inconvenient occupants. That is the analogy: an ASI could have preferences far more complex than ours and still be arbitrary, from our view, in ways that clash with continued life. Then the usual comfort gets worked over. Maybe people stay useful, like horses once did. But the comparison frays when better technology replaces weak biological labor. Usefulness can expire. Motor cars, trains, tractors, and lab-grown meat show how practical animals get outclassed. If bodies are slow, error-prone, sick, and costly to maintain, superintelligence may simply build around them. The close turns to politics: survival would need strange coalitions, because waiting for every faction or nation to agree on everything else is its own way of freezing.

As heard by us

A serious AI-risk episode that turns a huge warning into a clear, grounded argument.

The episode treats the case for artificial superintelligence as a warning, not a stunt. It starts with the 2023 open letter and then follows the book by Eliezer Yudkowsky and Nate Soares at MIRI, where the core claim is stark: staying on the current path could end in human…

Read the full review in PlayNext →

Why you'd press play

When you want MIRI's warning about ASI laid out without a safety rail.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Branches of Philosophy Podcast