Deploying GenAI models is easy—governance, cost control, and scale are the hard parts. If you're a platform engineer, API architect, or developer enabling AI in Kubernetes, you’ve likely hit issues like authentication, rate limits, PII redaction, traffic routing, and usage tracking. These are API problems, and the solution is an AI Gateway.

In this session, we’ll explore how AI Gateways act as policy and control layers for LLMs and inference workloads. Using CNCF tools like KServe, Envoy, Kuma, Dapr, OPA, Prometheus, and Backstage, we’ll show how to manage model access, enforce rules, expose internal AI APIs safely, and enable developer self-service.

You'll walk away with patterns and practices that help you scale AI responsibly, without reinventing the wheel