As Generative AI models move into production, traditional observability metrics fall short in capturing model-specific risks such as hallucination, toxicity, or bias.
This session explores how to extend open-source observability stacks—Prometheus and Grafana—to monitor GenAI systems at scale. We’ll discuss designing exporters and dashboards that track model health, including hallucination and bias detection, token-level usage metrics, and request performance (volume, latency, success rate).
Attendees will learn practical patterns for instrumenting LLMs, embeddings, and Model Control Planes (MCPs), integrating model evaluation signals into observability pipelines, and visualizing drift or degradation trends over time. The talk demonstrates a unified approach to trust, transparency, and performance analytics for both new and existing GenAI workloads using cloud-native tools.