Weizhou Lan
daocloud network cloud-native
Shanghai, China
Actions
I am a Senior Tech Lead at DaoCloud with over 15 years of engineering experience. I am an active member of the CNCF community across Kubernetes, Kubernetes SIGs, Cilium, llm-d, and Istio, and a CNCF maintainer of Spiderpool, a CNCF Sandbox project. I have served on the KubeCon Program Committee four times, delivered 10 KubeCon talks, and led the development of over five cloud-native open-source projects. My expertise spans networking, eBPF, observability, service mesh, and AI infrastructure.
Badges
an innovative network solution of underlay CNI
The underlay CNI solution can meet the needs of low-latency applications, some traditional host applications transporting to the cloud, and storage components requiring independent network to transmit datas.
Currently, there are not many options in the open source community. CNI projects such as macvlan SRIOV ipvlan miss functions like IPAM, clusterip communication. Projects like multus can insert multiple network interface to pod, problems such as routing tuning and IP allocation among multiple network interfaces cannot be solved. There is no IPAM projects in the community to support complicated requirements of underlay IPAM, such as IP dual-stack support, fixed IP for application, IP recycling, IP conflicts, etc..
Spiderpool give IPAM solution, and could integrate macvlan, SRIOV, ipvlan, multus to solve above-mentioned pain points and well integrate various infrastructure resources. It results to a new Underlay network solution.
【Sponsor】RDMA 容器网络下的大规模AI训练探索
随着Chat-GPT点燃了人们对于AI的热情和期待，各个厂商、开发者们开始日益关注AI领域，大模型的一次训练可能需要耗费数十天，因此，解决好在AI训练过程中的算力调度和网络通信，能提高训练的成功率，降低训练时间。本次分享中，首先介绍了大规模训练情况下可能会遇到的网络问题，其次，通过使用提供 RDMA 能力的 spiderpool kubernetes CNI插件，可以满足在各种大规模多机多卡分布式训练的 underlay 和 RDMA 网络要求，友好的支持ROCE 和 InfiniBand 场景，大大提高了分布式训练的通信效率。最后，我们探索了大模型在多个集群上训练的可行性，介绍了我们的多集群训练和多集群网络的方案。
Cloud Native Networking For AI : Strengthen CNI For RDMA
With the popularity of running AI workloads on kubernetes, RDMA technology with high network performance and CPU offload features helps save the training time. Only SRIOV, macvlan, and ipvlan technologies can effectively demonstrate RDMA capabilities, but in practice with Nvidia's network-operator, there is many problems,for example, unable to access clusterIP, lacking stable IPAM, and unable to manage the routing of multiple network cards effectively.For simultaneously accommodating the network requirements of both AI and traditional applications, in many practices, it has to implement dual CNIs. This presentation provide a solution to make macvlan and SR-IOV satisfy all needs with single CNI.
Per-node Api-server Proxy: Expand The Cluster's Scale and Stability
For lots of CNCF projects, kinds of daemonsets simultaneously synchronize datas from the Api-server from each node. Especially in large-scale clusters, it creates significant pressure on the Api-server, burdens the network, even affects the stability of the cluster.
Some projects have implemented optimization to address this. For instance, Cilium aggregates endpoint information into the CRD CiliumEndpointSlice before distributing it to its daemonset. However, many projects have not yet adopted such data aggregation optimizations and Currently, there is still no project to help improve the communication between all components and the Api-server.
ClusterPedia supports to launch per-node Api-server proxies to serve all local pods, and utilize eBPF to resolve the API server's clusterIP to the local proxy, which transparently implements API server access redirection on demand. In large-scale clusters, this can significantly improve the stability of all cluster's services.
Closing the Gap: Fair North–South Bandwidth Management for Tenants
In multi-tenant Kubernetes clusters, enforcing unified north–south ingress bandwidth limits is essential to ensure fair resource sharing. Although this sounds straightforward, Kubernetes currently lacks an effective solution.
Most CNI plugins only support per-Pod ingress bandwidth control. As workloads scale out with multiple replicas, it becomes impossible to enforce consistent limits at the tenant or application level. On the other hand, community LoadBalancer implementations do not provide ingress bandwidth management for LoadBalancer Services. As a result, operators often have to rely on external components outside the Kubernetes.
In this talk, we will present a lightweight, in-cluster approach built on Cilium CNI combined with Spiderpool. This design enables native bandwidth limiting for LoadBalancer Services, making it possible to achieve tenant-level ingress bandwidth management within Kubernetes, without adding external complexity.
Cybertwin-based Cloud Native Network (CCNN): Network Architecture Innovation and Practice
Driven by IoT and AI, future networks demand an architectural revolution. A CCNN is proposed that converges RAN, IP bearer and data center networks, using Kubernetes as the network OS to unify virtualization, scheduling, and management of compute, storage and network resources, thus enabling secure, cross-domain resource integration. The demo comprises: a Cybertwin-based personal agent service; the security agent function of Cybertwin that implements zero-trust; the data agent function that provides secure multi-dimensional personal data services; the communication agent function that delivers a cloud-native wireless access method for users; finally, a distributed medical collaborative inference based on the user’s personal agent, which demonstrates application-centric unified scheduling and intelligent orchestration of transmission, computing, and storage resources, laying an original, cutting-edge foundation for resource networking and intelligent scheduling in cloud-native networks.
Enhanced Service Redirection: eBPF Ensures The High Availability of Node-Local DNS
Excessive DNS query pressure can cause CoreDNS failures, leading to a cluster-wide meltdown. Thus, per-pod or per-node DNS proxy is a common high-availability solution. By binding the clusterIP of the kube-dns service to the local node, node-local DNS can provide seamless DNS interception and proxying, meeting the demands of low latency and high throughput. However, proxy's breakdown can directly lead to access failures, lacking high availability.
To solve this issue, we use cgroup eBPF to implement enhanced service redirection for node-local DNS, inspired by Cilium's LocalRedirectPolicy. Based on the health status of the local proxy or the redirection QoS strategy, the query of the kube-dns service can be dynamically resolved to the local node-local DNS, otherwise normally forwarded to CoreDNS. This scheme is independent of the CNI, and could cooperate with kube-proxy, offer multiple service redirection strategies and provide production-grade quality assurance for node-local DNS.
Making Topology-Aware Scheduling Practical for AI Workloads: From Discovery to Simulation At Scale
In large-scale AI inference clusters, multi-tenant workloads require both efficient GPU utilization and dynamic RDMA networking. However, heterogeneous GPU interconnect technologies inevitably lead to multi-level network topologies, such as scale-up networks and RDMA spine–leaf structures.
These diverse topologies introduce several challenges: Dynamic topology discovery and health detection across multiple layers, including scale-up, RDMA spine, and RDMA leaf. Second, Topology-aware scheduling that supports priority-based placement and ensures GPUs leverage optimal communication paths.Third, Validation at scale, requiring cost-effective simulation of large, multi-level topologies instead of relying on expensive hardware.
In this talk, it will share practical approach of topology discovery to help Kueue to achieve topology-aware scheduling, and showcase how Kwok simulates thousands of virtual nodes with multi-level topologies, enabling large-scale validation at zero hardware cost.
Project Lightning Talk: Spiderpool Unlocks Fine-Grained RDMA Observability for AI Inference
AI inference jobs using models of various sizes may run concurrently on a node, and also introduce cross-node RDMA communication. The lack of RDMA observability means performance bottlenecks go unnoticed and hardware remains underutilized, and node-exporter’s provision of only node-level RDMA metrics can not meet these demands.
Recently, Spiderpool has unlocked full RDMA observability. Multiple RDMA devices via SR-IOV are assigned to each pod, each with its own fixed IP address. This enables each pod’s RDMA traffic to be visible even at the switch’s link level. Additionally, Spiderpool collects RDMA metrics for each pod and node, and provides multi-layer Grafana dashboards aggregating metrics at the node, AI job, and pod levels. Especially in multi-tenant AI inference clusters, this helps identify hot applications and detect communication bottlenecks. This talk will explain how the solution boosts troubleshooting efficiency by over 60% through practical cases.
RDMA Meets eBPF: Transparent Acceleration for TCP Applications
For network IO-intensive applications, common acceleration optimization methods have already pushed performance to a certain bottleneck. Is it possible to further improve network access performance using RDMA? By default, it must use the Libibverbs API for coding to leverage RDMA, which introduces unacceptable modification costs for some applications. Additionally, when accessing service by clusterIP, kube-proxy can not parse RDMA packets, and it is a big challenge to implement clusterIP parsing on RDMA devices.
This topic introduces a new scheme for deploying and implementing RDMA transparent acceleration for TCP applications. It involves allocating RDMA devices to pods, using SMC for RDMA transparent acceleration, and employing cilium with cgroup eBPF to implement parsing of clusterIP. Ultimately, TCP applications can achieve significant improvements in network transmission with zero modification costs. Testing has shown that Redis performance can be improved by up to 50%.
Spiderpool Updates For AI Workloads: DRA, NRI, and RDMA Observability
Currently, the RDMA device plugin cannot coordinate the allocation of PCIe-affined RDMA devices based on GPU device allocation and requires users to manually ensure consistency between RDMA resources and Multus network configurations in the Pod YAML. To address this, Spiderpool introduces DRA to declare RDMA resources and network interfaces, significantly simplifying the configuration complexity in Pod YAML. Additionally, during Pod startup, Spiderpool leverages NRI to dynamically insert RDMA devices with GPU affinity, ensuring optimal GPU Direct RDMA performance.
Furthermore, existing community projects lack support for exporting metrics for isolated RDMA devices based on SR-IOV. Spiderpool bridges this gap by exporting comprehensive RDMA metrics for both Pods and nodes to Prometheus. It enables RDMA observability from multiple perspectives, including AI tasks.
These features represent groundbreaking innovations within the community, effectively empowering AI workloads.
Spiderpool Updates: DRA Helps Allocate RDMA Devices With GPU Affinity For AI Workload
Currently, the CNI barely meets some increasingly prominent network demands. Especially AI workloads requiring network cards in buck, the node scheduling could not simultaneously fullfill requirements for the macvlan master interface, subnet availability, and RDMA resources. Additionally, the physical affinity between the allocated network cards and GPU is not guaranteed to achieve high-performance transfers such as GPUDirectRDMA. Therefore, many end users complain and have to write complex network configurations in yaml and simply insert all network interfaces for pods to ensure device affinity, which is rigid.
Based on the practice requirements of popular tail-optimized AI network topology, in latest version, Spiderpool introduces DRA to uniformly declare various network configurations, schedule AI workloads to proper nodes, and automatically allocate network interfaces, IP addresses, and RDMA devices with GPU affinity on demand. This greatly enhances the flexibility in AI clusters.
KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 Sessionize Event
KubeCon + CloudNativeCon Japan 2026 Sessionize Event
KubeCon + CloudNativeCon Europe 2026 Sessionize Event
CNCF-Hosted Co-located Events Europe 2026 Sessionize Event
KubeCon + CloudNativeCon Japan 2025 Sessionize Event
Project Lightning Talk + Maintainer Track + Contribfest: KubeCon + CloudNativeCon Europe 2025 Sessionize Event
KubeCon + CloudNativeCon North America 2024 Sessionize Event
CNCF-hosted Co-located Events North America 2024 Sessionize Event
Project Lightning Talk + ContribFest + Maintainer Track: KubeCon + CloudNativeCon North America 2024 Sessionize Event
KCD Shanghai 2024 Sessionize Event
CNCF-hosted Co-located Events Europe 2024 Sessionize Event
KubeCon + CloudNativeCon North America 2023 Sessionize Event
Weizhou Lan
daocloud network cloud-native
Shanghai, China
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top