Traditional application monitoring is not enough for systems where an LLM decides what to do next. Production AI agents require visibility into prompts, model responses, retrieval, tool calls, agent handoffs, latency, cost, failures, and evaluation results. This session introduces practical AgentOps patterns for Azure-based agentic applications. We will explore tracing complete agent workflows, evaluating non-deterministic outputs, monitoring tool execution, detecting regressions, measuring grounding quality, and integrating agent evaluation into delivery pipelines. Attendees will learn how to make agent systems observable, testable, and operable so production failures can be diagnosed rather than hidden inside a generated response.

Focus: Operating AI agents in production: tracing, evaluation, tool-call monitoring, reliability, testing, latency, cost, CI/CD, and incident investigation.