The explosion of Large Language Models (LLMs) into enterprise systems has accelerated adoption of cloud-native platforms for hosting, scaling and integrating AI-driven workloads.
Kubernetes, with its rich ecosystem of observability, policy and runtime controls, is increasing the default substrate for deploying LLM-powered applications.
But as these systems move from experimentation to production, there are certain concerns:
- How do we secure LLMs in Kubernetes?
- How do we make their behavior observable & accountable?

This panel brings together practitioners working at the intersection of AI/ML, Kubernetes, cloud-native security, and observability to explore what it takes to run LLMs responsibly at scale through:
- designing guardrails with best of CNCF tools
- extending observability frameworks such as OpenTelemetry and Prometheus for LLM workloads
- tackling the challenges of integrating LLMs into multi-tenant Kubernetes environments
- addressing emerging risks like adversarial prompts, data exfiltration & RAG poisoning