As organizations continue to migrate their workloads to Kubernetes, they encounter a growing diversity of requirements among different application types. In particular, AI applications often require computing resources for efficient model training and serving. For years, the default setup involved virtualization, offering enhanced server utilization. Deploying Kubernetes on virtual servers seemed like a natural choice, leveraging the flexibility of virtualization. As the demand for resources increases the use of bare metal servers becomes more attractive.
This session will outline a journey of transitioning from virtual machines to bare metal servers. While virtualization abstracts away some complexities, moving to bare metal requires a deep understanding of CPU process scheduling, memory allocation, and more.
During the migration journey, we shifted our AI cluster from virtual machines to bare metal, only to encounter a surprising challenge: our workloads were running 10 times slower.