Thoughtworks Technology Podcast · Thoughtworks

Caring about documentation in the LLM era (w/ Heidi Waterhouse)

·39 min·2 clips

For years you could not search Microsoft's help site for 'blue screen of death,' because Microsoft called it a 'critical error message.'

1. The Thoughtworks Technology Podcast episode on caring about documentation in the LLM era covers how generative AI changes what technical writers do, using examples from OpenAPI to Microsoft's help site. 2. Host Lily Ryan interviews Heidi Waterhouse, a technical writer who wrote Docs for Developers in 2021 and has Progressive Delivery coming out in November 2025. 3. Waterhouse's thesis is that documentation is 'the way we describe software to humans,' and that LLMs are safest when retrieving from well-structured sources rather than guessing at intentions. 4. She says an OpenAPI-described API can safely sit behind a chat interface, but a request like 'write me a standard corporate dress code' would miss country, formality level and gender issues. 5. She separates a concordance, which only maps where a word like 'goats' appears in the Christian Bible, from a human-made index, and cites the indexer in Kurt Vonnegut's Cat's Cradle who says an index gives 'the shape of an author's mind.' 6. She describes Confluence search as indexing everything with 'no semantic weighting,' and contrasts it with tools like Glean that rank results by team proximity and recency without generating text. 7. Ryan raises package maintainers who receive complaints about features that do not exist because users read the docs through an LLM, and Waterhouse says she is not precious about wording but worries about whether truth survives the translation. 8. Waterhouse proposes a 'fantasy dial' on which she wants documentation retrieval at about 20% fantasy, smoothing text segments together without inventing anything, and she praises natural-language queries such as 'how do I' as 'immensely powerful.' 9. She recommends keeping a personal file of failed prompts as 'experimental data,' and says that naming the audience unsticks her about 80% of the time when she cannot write a document. 10. Suited to developers, technical writers and AI practitioners who write prompts.

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Thoughtworks Technology Podcast