Practical AI · Practical AI LLC

Video generation with realistic motion

January 23, 2025·45 min·4 clips

Early video AI made humans levitate instead of walk, exposing fundamental gaps in motion understanding.

Paras Jain and his brother A.J. Jain founded Genmo in December 2022 after completing PhDs at UC Berkeley, where Paras worked on distributed systems in the RISELab (the lab behind Apache Spark and Ray) and A.J. co-authored the DDPM paper that became the foundation for modern diffusion-based image generation. They chose video generation because it seemed impossible in 2022. Jain explains that the fundamental challenge of video generation is not just that video files are 100x larger than images: the harder problem is that most internet video does not actually teach a model about the physical world, because most footage consists of static objects or talking heads. Training a good video model requires curating data that teaches physics — how objects move, how fluid behaves, how inertia works — not just visual patterns. Early video generation models could not produce realistic walking; Genmo's own model Replay made humans hover above the ground like hovercrafts. Their third foundational model, Mochi, is described as the current state-of-the-art open-source video generation model, ranking neck-and-neck with closed models on the Artificial Analysis leaderboard based on Elo-style human preference ratings. Jain discusses evaluation infrastructure, explaining that human preference is a flawed benchmark because people reliably choose higher-resolution video over physically accurate motion, even when accuracy would matter more in production contexts. He describes functional test cases — such as a woman drinking from a glass with ice cubes, designed to detect whether the model understands that water must go into the glass rather than through its side — borrowed from his experience in self-driving perception safety testing. OpenAI's Sora is mentioned as a watershed moment for public awareness of video generation. Jain says Genmo open-sourced Mochi specifically to lower the infrastructure barrier for new entrants who would otherwise need to build petabytes of data pipelines before fine-tuning. The tone is technical but accessible, sitting at the intersection of ML research and product development. Suited to AI engineers, researchers, and practitioners interested in generative video systems.

As heard by us

Video generation is treated as a motion problem, with Genmo's architectural evolution front and center.

Video generation is treated here as a motion problem, not a novelty. Daniel Whitenack and Chris Benson speak with Paras Jain of Genmo about why motion sits at the center of the work, how that idea keeps resurfacing at Genmo, and how three large pre-training runs have shaped its…

Read the full review in PlayNext →

Why you'd press play

You want the real bottleneck in AI video, not a demo reel.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Practical AI