# State-of-the-Art KV Transfer for Disaggregated LLM Serving in vLLM

PyTorch Conference North America 2026 · [All sessions](https://sessionize.com/api/v2/s2rq62nk/view/markdown)

- **Session ID:** 1255023
- **When:** Tuesday, October 20, 2026 2:15 pm - 2:40 pm (Pacific Standard Time, UTC-08)
- **Room:** LL20C (Lower Level)
- **Speakers:** Nicolò Lucchesi, Sunita Nadampalli, Zhanqiu Hu

## Description

As LLM deployments scale, disaggregated P/D serving is emerging as a key architecture by separating compute-bound prefill from memory bandwidth-sensitive decode. An efficient KV cache transfer layer is critical to preserve disaggregation's benefits, and a flexible interface is necessary to generalize across models and heterogeneous hardware such as NVIDIA GPUs and AWS Trainium.
Agentic workloads with multi-turn conversations require state to flow bidirectionally between P and D, while production systems must remain resilient to failures and traffic spikes.
Hybrid models such as Nemotron, Qwen3.5 and DeepSeek-V4 interleave attention layers or combine KV compression and sparse sliding-window, producing diverse cache layouts and sharding semantics that interact with optimizations like prefix caching and CPU offloading.
We present developments in vLLM's disaggregated serving stack that address these challenges, including hybrid model transfer with heterogeneous TP, bidirectional KV transfer, KV Push connector, and KV cache leases for reliability. KV Push improves responsiveness by reducing TTFT, while on Nemotron disaggregated P/D Pareto-dominates co-located across concurrency levels.

## Speakers

- [Nicolò Lucchesi](https://sessionize.com/api/v2/s2rq62nk/view/markdown/speaker/c44bac6e-1d65-4d15-9729-5dfc693a1a89) — Research Engineer, Mistral AI, vLLM Maintainer
- [Sunita Nadampalli](https://sessionize.com/api/v2/s2rq62nk/view/markdown/speaker/31403442-97e0-4ecf-a4b0-a46b13af65f6) — Principal Engineer, AI/ML expert at Amazon
- [Zhanqiu Hu](https://sessionize.com/api/v2/s2rq62nk/view/markdown/speaker/9a8cea08-5e46-4ff5-9dc2-c288d9d6d4a8) — Machine Learning Engineer, Red Hat