Building production-grade AI infrastructure at scale requires more than provisioning GPUs — it demands a unified, automated platform spanning bare metal, VMs, and Kubernetes under a single tenant-aware fabric. This talk presents the engineering journey of FPT AI Factory in operating one of Southeast Asia's largest GPU clouds: ~3,000 NVIDIA H100, H200, A100, and A10 accelerators across three regions in Vietnam and Japan.

The session covers four pillars: (1) automation and multi-tenancy on InfiniBand and RoCE fabrics with PKey isolation and topology-aware scheduling; (2) a unified VPC model that bridges bare-metal, VMs, and Kubernetes via customized OVN as the network gateway; (3) managed Kubernetes services with GPU operator, MIG slicing, and NCCL-tuned scheduling; (4) production observability built on Prometheus, DCGM, and custom fabric exporters. Attendees will leave with actionable patterns from running 3,000 GPUs across three regions in production.