Software Engineering Institute (SEI) Podcast Series · Members of Technical Staff at the Software Engineering Institute

Using Role-Playing Scenarios to Identify Bias in LLMs

·45 min·4 clips
ChatGPT assigned Hispanic names as ranch hands and non-Hispanic names as astronomers, exposing hidden stereotypes in AI.
SEI researchers Katie Robinson and Violet Turi discuss their experiment using role-playing scenarios to uncover bias in large language models like ChatGPT. They found stereotypes in role assignments based on gender and ethnicity, highlighting risks in real-world applications and the need for creative auditing techniques.

As heard by us

A focused look at how name-and-role oddities can surface bias in LLMs.

The episode treats bias in large language models as an engineering problem with real trust consequences, and it does so by grounding the discussion in the work of SEI researchers Katie Robinson and Violet Turi.

Read the full review in PlayNext →

Why you'd press play

Listen if you want a practical look at how bias shows up in large language models.

Read the full recommendation in PlayNext →
Listen to the show on