Prompting a frontier LLM to flag bias in news works well in a demo. In production, with articles arriving continuously from publishers and social media, the cost, latency and inconsistency soon become a problem. This talk describes how we built a system that classifies around 70 types of bias (from framing and omission to loaded language and ideological slant) and rewrites stories to be balanced. We began with LLM prompts, used their outputs plus human review to build a large, politically balanced dataset of news stories, and trained an in-house model that replaced most of the API calls.

You'll see how we designed the taxonomy, how we kept the dataset balanced, where prompting still beats the small model, and the human-in-the-loop checks that stop "debiasing" from turning into censorship. I'll also cover how we measured cost, latency and agreement with human reviewers before and after the switch.

Key takeaways:
1. A practical path from prompt prototype to labelled dataset to an owned model
2. Designing and balancing a multi-label bias taxonomy
3. Where to keep humans in the loop and how to measure fairness