Session

Designing LLM APIs That Don't Hallucinate Your KPIs

We launched an LLM-powered analytics assistant that let business teams query their KPIs in plain English. Two weeks in, a VP flagged that revenue numbers didn't match the dashboard. Close — but not right. The model was generating SQL with a slightly different date grain than our canonical KPI definitions. No errors. No warnings. Just confidently wrong numbers in front of an executive. That's how LLM analytics tools fail: not obviously, but in ways that quietly destroy trust and kill adoption. The problem had nothing to do with the model. It was in how we designed the API layer connecting the LLM to our data systems — a mistake we've since seen replicated across many teams building similar tools. This session diagnoses the API-layer failures that cause LLM analytics to produce confidently incorrect outputs and presents the architecture patterns that prevent them. We cover KPI definition schemas explicit enough to eliminate LLM ambiguity, validation layers that catch semantic mismatches before results reach users, and fallback path design for queries the model shouldn't answer alone. The 60-minute format gives us time to go deep on the governance side: defining KPI ownership clearly enough for an LLM to use reliably, building feedback loops that surface hallucination patterns before they become trust problems, and structuring the collaboration between data engineering and the LLM application layer so the system stays trustworthy as both the data model and the underlying model evolve.

Anusha Kovi

Business Intelligence Engineer, Amazon

Seattle, Washington, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top