Session

Streaming Was Only Half the Fix: Retrofitting a Stable gRPC API

The Kubernetes Container Runtime Interface is a gRPC API on every node. Its list operations used unary RPCs. That worked until nodes accumulated thousands of completed and failed containers. At that density, responses cross the receive limit, producing ResourceExhausted errors. Raising the limit only postpones the failure and does nothing about unbounded result sets.
The fix was adding six streaming RPCs to CRI and implementing them in CRI-O, with UNIMPLEMENTED-based fallback to unary during mixed-version rollout.
But streaming did not fix everything. gRPC provides flow control through HTTP/2, but the first implementation gathers the full result set before calling Send. Messages are smaller on the wire, but server memory stays roughly the same because flow control cannot help if everything is materialized first.
This case study from the CRI-O maintainer covers why unary broke, how rollout worked, and what pipeline streaming requires on the server side.

Sohan Kunkerkar

Senior Software Engineer at Red Hat

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top