# Demo Theater: OpenCLIC: Open Communication for Distributed Inference

PyTorch Conference North America 2026 · [All sessions](https://sessionize.com/api/v2/s2rq62nk/view/markdown)

- **Session ID:** 1329720
- **When:** Wednesday, October 21, 2026 2:30 pm - 2:40 pm (Pacific Standard Time, UTC-08)
- **Room:** Community Expo (Concourse Level)
- **Speakers:** Jesse Treger

## Description

Distributed inference is changing what AI frameworks need from RDMA networking. Mixture-of-Experts models repeatedly dispatch and combine tokens across GPUs, creating dynamic, latency-sensitive all-to-all traffic unlike the large-message xCCL collectives of distributed training. Current solutions lower latency through tightly integrated GPU, communication, and networking stacks built around connection-oriented RDMA transports, trading portability and developer choice for performance. These approaches remain a poor structural match for the many-destination communication patterns of expert-parallel inference, often consuming GPU execution resources for communication that could otherwise be devoted to inference.

A more open and scalable path starts from a different question: what should inference communication look like if it were designed for connectionless RDMA networks from the ground up, rather than adapted from connection-oriented ones? The emerging Ultra Ethernet Transport—modeled on the HPE Slingshot interconnect deployed in many of the world’s most powerful supercomputers—avoids per-peer connections to improve multipathing, loss recovery, and scalability. It provides an open foundation for innovation across accelerators, networks, and software.

This presentation will introduce OpenCLIC, an open inference communication framework for connectionless networks being integrated with vLLM. Expert-parallel dispatch and combine are its first use case, with broader support planned for one-sided operations such as KV-cache transfer, cache reuse and offload, and model-state movement. OpenCLIC separates framework interfaces, GPU kernels, and transport implementations so developers can use a common interface for adding communication-aware inference capabilities without tying them to a single accelerator or network stack.

Its initial HPE Slingshot implementation uses kernel-initiated OpenSHMEM communication to expose network resources directly to GPU code and keep the CPU out of the critical path. Designed to extend across accelerators and fabrics, OpenCLIC demonstrates how portable GPU-initiated communication and scalable connectionless networking can support expert parallelism and the data-movement needs of distributed inference. By separating communication algorithms from transport implementations,OpenCLIC preserves flexibility in hardware choices while creating a path for inference software to exploit more capable communication architectures as they emerge.

\*******
_In order to facilitate networking and business relationships at the event, you may choose to visit a third party’s booth or access sponsored content. You are never required to visit third party booths or to access sponsored content. When visiting a booth or participating in sponsored activities, the third party will receive some of your registration data. This data includes your first name, last name, title, company, address, email, standard demographics questions (i.e. job function, industry), and details about the sponsored content or resources you interacted with. If you choose to interact with a booth or access sponsored content, you are explicitly consenting to receipt and use of such data by the third-party recipients, which will be subject to their own privacy policies._

## Speaker

- [Jesse Treger](https://sessionize.com/api/v2/s2rq62nk/view/markdown/speaker/72169699-a457-48d4-9871-e0d1932bd9d0)