# Future of Distributed Communication in PyTorch: New APIs for Fault Tolerance, RDMA and Extensibility

PyTorch Conference North America 2026 · [All sessions](https://sessionize.com/api/v2/s2rq62nk/view/markdown)

- **Session ID:** 1256220
- **When:** Tuesday, October 20, 2026 4:20 pm - 4:45 pm (Pacific Standard Time, UTC-08)
- **Room:** LL21A (Lower Level)
- **Speakers:** Tristan Rice, Kapil Sharma

## Description

As training scales to hundreds of thousands of GPUs across heterogeneous hardware, PyTorch's distributed communication layer needs new capabilities: fault tolerance without job restarts, direct memory access across nodes, observable collectives, and plug-and-play backend registration.

This talk walks through the new APIs hands-on:

\* Fault Tolerance — torch.distributed.reconfigure() enables live process group reconfiguration after rank failures, keeping jobs running.
\* Window APIs — One-sided RDMA put/get for zero-copy communication for workloads such as RL.
\* Composable Hooks — Customizable logic and logging for all collectives, collective tracing, NaN detection, and more across all backends.
\* Backend Extensibility — New backends ship as pip-installable wheels via entry_points, acting as first-class citizens without patching PyTorch.
\* Clean Interfaces — Improved backend APIs lowering the barrier for advanced users and hardware vendors.

We've incubated these features in TorchComms and are now upstreaming them directly into torch.distributed so every user benefits with zero migration cost.

## Speakers

- [Tristan Rice](https://sessionize.com/api/v2/s2rq62nk/view/markdown/speaker/dde599d6-57ee-4fbc-9a07-ae71ff0c3c61) — Meta - PyTorch
- [Kapil Sharma](https://sessionize.com/api/v2/s2rq62nk/view/markdown/speaker/77fd5103-92a9-4c3e-a919-726a80e8f4bd) — Kapil Sharma, Meta, PyTorch Distributed