Most teams treat ML pipelines, AI batch jobs, and agent workflows as three separate infrastructure problems. They run ML training on one platform, AI inference on another, and emerging agent workflows on a third hastily-assembled stack. The fragmentation creates real cost: duplicated tooling, inconsistent observability, and infrastructure that cannot evolve as fast as the workloads on top of it.
This talk shares how we built a single unified orchestration layer to handle ML training, AI inference pipelines, and agent workflows at consumer scale - serving hundreds of millions of users downstream.

The first half walks through the architectural decisions: why a single orchestration layer wins over fragmented tooling, the type-safe task interfaces and container-native execution patterns that gave us the reliability guarantees production AI demands, the tradeoffs between adopting open-source workflow systems versus building proprietary tooling, and what we had to design on top of standard workflow primitives to handle non-deterministic agent execution that static ML pipelines never encounter.

The second half covers the production evolution: how the same infrastructure that originally served ML training jobs now routes and manages agent workflows in production. We cover the new failure modes that emerge with agent workloads, the retry strategies that broke down and what replaced them, the observability patterns required when execution paths are non-deterministic, the cost and resource scheduling implications, and the architectural patterns that make agent orchestration fundamentally different from pipeline orchestration.

Attendees leave with a concrete blueprint for designing a unified orchestration runtime for ML, AI, and agent workloads, an honest account of where this approach struggles, and a clear-eyed view of where the broader infrastructure ecosystem needs to evolve next.