Session
From Prompt to Production: Practical Observability for AI-Native Apps
Building an impressive AI demo is easy. Running AI-native software reliably in production is where things get hard. In this session, we will walk through a practical observability approach for applications built with the Microsoft Agent Framework, focused on what engineering teams actually need after launch: traceability, debuggability, and operational confidence. Using real-world architecture patterns, we will show how to instrument multi-step agent workflows, correlate prompts, tool calls, model responses, and business transactions, and detect failure modes that traditional APM misses. You will learn how to identify hallucination-adjacent behavior, spot context drift, track token and latency hotspots, and build meaningful alerts around agent reliability, cost, and user impact. This is not a beginner intro and not a theory talk. Expect concrete implementation guidance, telemetry design decisions, and production lessons learned, including what to measure first, what to ignore, and how to turn noisy AI traces into actionable signals. You will leave with an AgentOps playbook you can apply immediately to ship, monitor, and continuously improve AI-native applications.
Harry Kimpel
New Relic, Principal Developer Relations Engineer
Garmisch-Partenkirchen, Germany
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top