LLM services pose a new class of observability problems. Non‑deterministic latency, cost per token, and hallucination or quality metrics that conventional tools often miss.
By wrapping every LLM request in OpenTelemetry traces, metrics, and logs and running those instrumentations in a production Kubernetes environment, users can surface cost and latency signals, flag hallucinations in real time, and optimize resource usage across clusters and applications.
In this talk we'll walk through the SIG‑approved OpenTelemetry semantic conventions for GenAI, demonstrate a ready‑to‑deploy Kubernetes observability stack, and present case studies that show how observability turned an otherwise opaque AI service into a measurable, reliable component of a cloud‑native platform.