Session
Collisions in the Dark: Illuminating the 95% of CERN's Computing Platform You Can't See
Ask three High Energy Physics scientists what they need from monitoring and you'll get a superposition of completely different answers: “Why is my training slow ?”, “Are we maxing the rack’s power limit ?” or “Do we need more GPUs ?”
At CERN, as we were scaling the usage of the scientific computing platform, a new challenge emerged: gaining deep, actionable insights into the performance, resource consumption, and overall health of the entire system.
This talk presents our journey in building a comprehensive observability stack for our scientific computing platform from the ground up. We'll demonstrate how we leveraged the battle-tested monitoring tooling and created missing components to gain insights into the platform’s workloads. We will showcase the architecture of our observability stack, as well as the visualisations we designed to serve two key perspectives: the researcher who needs granular metrics like GPU utilization, I/O, and power draw for their specific workloads, and the Platform Manager who requires high-level views of usage patterns, idle resources to reclaim, cost of operations, availability of accelerators and allocation of compute over time.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top