Open Source Security Summit + AI_dev

eBPF-Powered Observability for Sustainable Computing - Paras Mamgain, Anmol Krishan Sachdeva, Samrat Priyadarshi & Miki Katsuragi, Google

With the advancements in the fields of Applied AI/ML, computational power, FinOps practices, and cloud services, organizations are increasingly building and deploying AI/ML workloads on Kubernetes. Cloud Computing gives the impression that the underlying resources (like Virtual Machines, CPU, RAM, Storage, etc.) are infinite, and hence many individuals don’t focus on the count of machines and resources when training their AI/ML models and proceed with training fairly large models with millions and billions of parameters. The energy-intensive nature of model training and iterations, coupled with the growing scale of AI/ML deployments, is contributing to a considerable environmental impact in the form of Carbon Emissions.

This talk introduces some key open-source cloud agnostic tools and techniques to build eBPF-powered observability solutions for practicing Sustainable Computing. By analyzing and tuning performance and energy metrics from Kubernetes resources like Pods and Nodes, AI/ML Practitioners and Platform Engineers can optimize workloads, reduce energy consumption, lower costs, and minimize environmental impact while maintaining performance.