Session
Testing for the Failures That Models Don't Cause
Most AI agent failures in production get blamed on the model. In practice, the data pipeline is often the culprit—and standard AI monitoring stacks are blind to it. They track model accuracy and infrastructure utilization. They don't track data freshness, event sequence completeness, ghost records, silent schema changes, or cross-agent coordination conflicts.
This talk walks through five specific detection techniques that expose the failure modes hiding beneath the model layer. How to measure the time delta between an event's occurrence and the agent's receipt of it. How to verify event sequence completeness across batch window boundaries. How to correlate agent actions against change data capture logs to detect decisions made on deleted records. How to catch silent schema changes before they degrade agent decisions. And how to cross-reference agent action logs to identify conflicting remediations before they cascade.
Each technique is presented with the failure mode it detects, the signal it looks for, and the open-source tooling that implements it. Attendees will leave with a testing framework they can integrate into their existing evaluation pipelines immediately.
Shazia Hasnie
VP Product Strategy & Innovation at Cuber AI
Los Angeles, California, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top