Containers are the best way to run machine learning and AI workloads in the cloud. However, running these workloads efficiently poses unique challenges, from resource management to performance optimization.

In this talk, we dive into the details of how GPUs are made available to such workloads when running with both standalone containers as well as with Kubernetes. As part of this, we discuss various options for sharing GPUs between them. These techniques include simple time-slicing, MPS, and MIG.

By the end of this session, attendees will have a comprehensive understanding of how GPU support in containers and Kubernetes works under the hood, as well as the knowledge required to make the most efficient use of GPUs in their own applications.