AI inference jobs using models of various sizes may run concurrently on a node, and also introduce cross-node RDMA communication. The lack of RDMA observability means performance bottlenecks go unnoticed and hardware remains underutilized, and node-exporter’s provision of only node-level RDMA metrics can not meet these demands.

Recently, Spiderpool has unlocked full RDMA observability. Multiple RDMA devices via SR-IOV are assigned to each pod, each with its own fixed IP address. This enables each pod’s RDMA traffic to be visible even at the switch’s link level. Additionally, Spiderpool collects RDMA metrics for each pod and node, and provides multi-layer Grafana dashboards aggregating metrics at the node, AI job, and pod levels. Especially in multi-tenant AI inference clusters, this helps identify hot applications and detect communication bottlenecks. This talk will explain how the solution boosts troubleshooting efficiency by over 60% through practical cases.