# Native Tiered KV Cache Offloading in vLLM: From Storage Offloading to Disaggregated Serving

PyTorch Conference North America 2026 · [All sessions](https://sessionize.com/api/v2/s2rq62nk/view/markdown)

- **Session ID:** 1254938
- **When:** Tuesday, October 20, 2026 2:50 pm - 3:15 pm (Pacific Standard Time, UTC-08)
- **Room:** LL20C (Lower Level)
- **Speakers:** Or Ozeri

## Description

As LLM applications shift toward long-lived agentic sessions, KV cache management becomes a critical scaling challenge. Sessions may remain idle for hours or days before a follow-up task arrives, yet their KV cache must be preserved. Offloading to cheaper storage tiers is essential to support more concurrent sessions cost-effectively. Beyond offloading, moving KV cache between instances enables load balancing and disaggregated prefill/decode.

We present vLLM's native tiered KV cache offloading framework, newly integrated upstream with no external dependencies. The key insight is routing all transfers through CPU memory as a universal transport hub, which yields multiple benefits: (1) minimal GPU overhead—each GPU performs only a simple copy to host memory; (2) consolidated I/O from a single CPU buffer rather than scattered across HBM devices; (3) hardware-agnostic design requiring no specialized transfer APIs; and (4) independence from KV cache memory layout, which varies across hardware, attention backends, parallelism schemes, and model architectures.

Join this talk to learn how to deploy, configure, and extend the framework for your own infrastructure.

## Speaker

- [Or Ozeri](https://sessionize.com/api/v2/s2rq62nk/view/markdown/speaker/518cd396-541e-4f05-a8fd-4c2491d1cb8f) — IBM, Cloud Researcher