AI agents are no longer just chatbots—they're autonomous systems making decisions, calling APIs, and orchestrating complex workflows without human intervention. But here's the thing: when you give an AI agent the keys to your infrastructure, how do you know it won't drive off a cliff?

In this session, I'll share real-world lessons from building and deploying agentic AI systems in production. We'll start with a story that keeps me up at night—an agent that seemed perfect in testing but made a series of cascading decisions in production that nobody anticipated. That incident taught me that traditional ML monitoring isn't enough when your AI can take actions on its own.

We'll dive into three critical areas:

Evaluation: How do you test an agent that can take thousands of different paths? I'll show you practical frameworks for evaluating agent behavior, including multi-step reasoning validation, tool-use accuracy, and decision quality metrics that actually matter.

Observability: When things go wrong (and they will), you need to understand why. We'll explore tracing techniques for multi-agent workflows, real-time monitoring patterns, and how to build audit trails that help you debug agent decisions after the fact.

Governance: This isn't about compliance checkboxes—it's about building guardrails that let your agents innovate safely. We'll cover bias detection in autonomous decision-making, implementing human-in-the-loop patterns where they matter most, and creating governance models that scale with your agent deployments.

You'll leave with practical patterns you can implement immediately, whether you're building your first agent or scaling a fleet of them in production.