Post-Training Infra for Everyone Else

Everyone wants to post-train agents now, but the infrastructure story is still shaped around expensive, compute-heavy workloads: frontier coding agents, long-running containers, and clusters where environment cost is a rounding error.

That is not where most companies will start.

A lot of teams will want to train, fine-tune, or benchmark narrow agents for their own workflows: customer support agents, internal ops agents, sales agents, workflow automation agents, repo-specific coding agents. These environments are often not CPU-bound. They spend most of their time waiting on LLM calls, APIs, databases, queues, filesystems, and reward checks.
So why does every environment need to be an always-on container?

This talk introduces Flarbor (the name is subject to change), an early experiment inspired by Harbor that asks whether RL environments for I/O-heavy agents can run on serverless primitives instead. Flarbor uses Cloudflare Durable Objects for stateful environment lifecycles, Project Think for the agent harness, `@cloudflare/shell` for workspace/filesystem support, Dynamic Workers for sandboxed code execution, and containers only as an escape hatch when the environment actually needs a Linux toolchain.

I’ll walk through what an RL environment looks like in practice: tasks, trials, tools, state, trajectories, reward functions, retries, and batch benchmarking. Then I’ll show where a Durable Object-based design changes the cost and scaling model: hibernation while waiting on model responses, cheap parallelism, durable state without managing a container per trial, and faster startup for lightweight environments.

This is not a claim that containers are bad. Harbor is a great model, especially when the environment really needs OS-level execution. The point is that containers should not be the only default. If post-training becomes something every company does for its own agents, we need environment infrastructure that is cheap enough and parallel enough for everyone else too.