TechSurge: Deep Tech Podcast · Celesta Capital | Deep Tech Venture Capital Firm

Understanding the Elegant Math Behind Modern Machine Learning

·1 hr 15 min·5 clips

Gradient descent walks down the slope of model error until the prediction improves.

1. TechSurge: Deep Tech Podcast frames “Understanding the Elegant Math Behind Modern Machine Learning” around the math behind modern AI systems like GPT and other large language models. 2. Shiram hosts the discussion with Anil Ananthaswamy, an award-winning journalist and writer whose latest book is Why Machines Learn: The Elegant Math Behind Modern AI. 3. The episode asks why the title says “Why Machines Learn” instead of “How Machines Learn,” and the answer is that the “why is hidden in the math.” 4. Anil says he was a software engineer before becoming a journalist covering particle physics, cosmology, neuroscience, and quantum mechanics. 5. He says machine learning started showing up in more of his stories around 2015 and 2016, which made the field feel like something he could understand viscerally. 6. He describes the 2019 Knight Science Journalism Fellowship at MIT as the moment he went “back to school,” taking classes at Harvard and MIT and relearning Python. 7. He says the project turned from code mechanics into a search for “the simplicity and the beauty of the math.” 8. The perceptron convergence proof from Frank Rosenblatt becomes the first major example of why the math matters. 9. Rosenblatt’s perceptron is described as a single-layer neural network that classifies two categories, such as dogs and cats, when they are linearly separable. 10. The conversation says Minsky and Papert’s 1960s book Perceptrons proved the XOR limitation for a single layer and helped trigger skepticism about neural networks. 11. The episode places K nearest neighbor and Bayesian classifiers in the same mid-1960s to late-1960s era of alternative machine learning work. 12. Geoff Hinton is presented as the researcher who stayed convinced that neural networks mattered even when multi-layer training was not yet solved. 13. Bernie Widrow and Ted Hoff are discussed as the Stanford pair who invented the LMS algorithm and built a working single artificial neuron in hardware over a weekend. 14. The speaker explains LMS as a noisy precursor to gradient descent, where the model moves downhill on an error surface toward a minimum. 15. Gradient descent is explained with a bowl-shaped error function and the “terraced rice paddy” image of stepping down a slope one level at a time. 16. Backpropagation is then defined as propagating blame from outer layers back through hidden layers so deep neural networks with one or more hidden layers can be trained. 17. The episode says Yann LeCun’s convolutional neural network work added translational and rotational invariance for image recognition. 18. It then connects Fei-Fei Li’s ImageNet dataset to the 2011 or 2012 AlexNet breakthrough by Hinton, Ilya Sutskever, and Alex Krizhevsky. 19. The tone is technical but conversational, with repeated analogies, brief historical detours, and a teaching style aimed at listeners who remember high-school math. 20. Listeners who like AI history, optimization, and neural network architecture will get the most from it, while people wanting a light nontechnical chat may skip it.

As heard by us

A clear look at machine learning's move from linear limits to modern AI.

TechSurge treats machine learning as a sequence of threshold crossings, from single-layer models that could only draw straight-line boundaries to deeper networks that can separate nonlinear patterns.

Read the full review in PlayNext →

Why you'd press play

Hear how neural networks evolved into the architectures behind modern AI.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from TechSurge: Deep Tech Podcast