Samzong Lu
PM at DaoCloud, AI/LLMOps PM Leader, CNCF Multiple Project Contributors, Open Source Enthusiast
Shanghai, China
Actions
- Samzong is Product Manager (Focus on AI/LLMOps, Muti-Cluster, Cluster LCM, Microservice, ServiceMesh )
- Kubernetes / Kubernetes-sigs active Contributor
- Karmada active Contributor, member
- Istio active Contributor, member
- CNCF Multiple Project Contributors
- CNCF Open Source Enthusiast
Area of Expertise
Topics
Exploring and Solving Challenges in Multi-Cloud, Multi-Cluster Environments with Karmada
More and more enterprises have to deal with increasingly complex business scenarios, and implementing multi-cluster applications can greatly improve the stability and security of application programs. So, how do you manage multiple Kubernetes clusters simultaneously and avoid vendor lock-in? reduce the additional costs associated with inconsistent application delivery in a multi-cluster environment? unify multi-cluster deployment, cross-cluster traffic governance, and security governance for your application programs?
In this session, we will introduce solve these problems using the Karmada project. You will learn how to achieve consistent application delivery in a multi-cluster scenario, unified deployment of application programs, automatic distribution, automatic scaling and fault migration of application programs, cross-cluster dr. during the tutorial, you leverage its functionality to solve various challenges encountered in actual business scenarios.
Open Source to Enterprise: Scaling LLM/Diffusion Model Inference in Kubernetes
Our session will unveil how Kubernetes-based cloud-native technologies power the transformation of cutting-edge LLMs and diffusion models from lab experiments to massively scalable SaaS services. Key highlights include:
1. Cloud-Native Scaling for AI Inference: Containerized deployment, dynamic scaling, and distributed scheduling on Kubernetes support millions of daily inference requests, with GPU utilization boosted by 40%;
2. Efficiency Breakthroughs in Inference: Through model quantization, distributed parallelism, and caching strategies, we achieved a 60% reduction in LLM inference latency and 35% cost savings for video generation;
3. SaaS Productization Journey: From API design to billing systems, learn how we packaged complex inference technologies into user-friendly services, driving 300% user growth and serving 500+ global enterprise clients;
4. Battle-Tested Solutions: Lessons from multi-model deployment and multi-tenant isolation scenarios, with open-source toolkits and reusable architecture templates for the community.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top