Session

Engineering the Reasoning Audit for the 2026 Agentic Economy

In 2026, we have a trust problem, as AI agents transition from "chatting" to "acting" within sovereign borders.
An unmonitored agent isn't just a technical bug, it’s a liability and a potential $20M compliance breach. The "vibe check" is officially dead, replaced by the necessity of rigorous, automated, and scalable eval frameworks.
Drawing from my experience building reasoning engines contributing to more than $100M cost savings for Microsoft, this session moves past static benchmarks to the tactical "How" of Agentic MLOps. We will deconstruct the architecture of domain-specific custom evaluations applied across two critical tracks: Real-Time Guardrails for immediate reasoning safety and Offline Deep-Dives for high-fidelity audits. I will share the blueprint for integrating "LLM-as-a-Judge" into CI/CD pipelines using specialized SLMs to ensure low-latency, cost-effective oversight without sacrificing data sovereignty.

Key Takeaways -
> The Hybrid Eval Stack: Audience will architect a Real-Time SLM judge for production safety and learn about frameworks for deep reasoning audits. Demo - How Logic Traceability could be implemented to audit multi-step task execution and catch the logical chain breaks before production roll-outs.
> Domain Specific Customization: Audience will get to know how to build custom evalsets tailored to specific industry use-cases and constraint, and leverage AI to generate thousands of adversarial edge cases that traditional training data misses.

Jatin Chawla

I am a Software Engineer at Microsoft (AI Commercial Engineering)

Bengaluru, India

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top