At last year’s FlinkForward in Seattle, we presented Flink Autoscaling (FLIP-271), a research-backed open-source autoscaling solution as part of the Flink Kubernetes Operator. From day one, users who adopted Flink Autoscaling have reported a significant reduction in resource usage, maintenance, and on-call burden.

But it’s not all sunshine and rainbows. Since its inception, Flink Autoscaling has gone through a serious of improvements and additions. The originally proposed idea of performing a deep introspection of the Flink deployment to find a stable and cost-efficient resource assignment hasn’t changed, but the robustness of the algorithm and its metrics have been greatly improved.

One year later, is is time to evaluate Flink Autoscaling and present some of the challenges alongside with the latest additions and changes. In the course of this talk, we will show the inner workings of the Flink Autoscaling and highlight some of its unique integrations with Flink.