Learn how traffic routing for ML Models poses unique challenges at internet scale, and how Netflix solves them through specialized software design patterns. Learn how various API Gateway solutions (REST), Service Mesh, and custom design can help scale traffic routing for efficient ML model serving. We will also briefly go into how various experiences on Netflix.com are powered by unique ML Models.
Here is the recent official blog post on this topic that I recently co-authored at Netflix.
https://netflixtechblog.com/state-of-routing-in-model-serving-16e22fe18741