Lenny's Podcast: Product | Career | Growth · Lenny Rachitsky

Inside the expert network training every frontier AI model | Garrett Lord (Handshake CEO)

·1 hr 10 min·5 clips

Handshake's PhDs break state-of-the-art AI models to generate data that has never been on the internet.

Garrett Lord explains that AI model training involves two phases: pre-training on the entire corpus of human knowledge from the internet, and post-training to improve model capabilities in specific domains. Around 18 to 24 months before the episode, labs hit diminishing returns on pre-training and shifted focus to post-training, which requires high-quality, expert-generated data across domains like advanced STEM, medicine, law, and finance. This created an explosion in demand for domain experts who can break models, find errors in reasoning steps, write correct ground-truth responses, and produce trajectory data showing step-by-step human problem-solving. Lord describes the key data types: supervised fine-tuning prompt-response pairs, reinforcement learning with human feedback preference rankings, trajectory data with screen recordings and voiceover narration, and rubric-based auto-evaluation environments. He explains that the models have gotten so good that generalist labelers are no longer sufficient, and labs need people who can genuinely challenge and break frontier models in specific domains. Handshake's competitive advantage is its decade of trust with 18 million active users who have verified academic credentials, eliminating the massive customer acquisition costs that plague competitors running targeted ad campaigns on Instagram and LinkedIn. Lord describes how the business began when third-party middleman companies started approaching Handshake to recruit PhDs, and Handshake realized it was generating tens of millions for those middlemen while its own users reported poor experiences on those platforms. The pivot crystallized when frontier labs started approaching Handshake directly, cutting out the middlemen. The company now works with seven frontier labs. Expert contributors, including PhDs working in their field of specialty, earn between $100 and $200 per hour. Lord addresses the tension between students training models that might reduce entry-level job opportunities, arguing based on actual employer data from one million companies using Handshake that AI is making young workers more productive rather than eliminating them. He gives examples of AI-native interns shipping production code on their first afternoon and solo social media managers replacing what previously required specialized teams.

As heard by us

Model quality still depends on people with real domain expertise.

Garrett Lord is presented as making a clear case for expert networks as part of AI infrastructure. The conversation stays close to Handshake's role as the largest expert network in the world, how it helps create new data after a model is trained, and why models may now need…

Read the full review in PlayNext →

Why you'd press play

You get a clear look at why experts still sit inside the AI stack.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Lenny's Podcast: Product | Career | Growth