Michas Szacillo
Senior Engineer @ Bloomberg
Actions
Michas is a senior software engineer and tech lead on Bloomberg’s Streaming Analytics engineering team. The platform, which is running on Kubernetes, serves as the foundation for many of Bloomberg's data streaming use cases. Michas is also a frequent collaborator to the CNCF community, with contributions to projects including Karmada and KServe.
Links
Multi-cluster Orchestration System: Karmada Updates and Use Cases
Karmada (Kubernetes Armada) is a Kubernetes management system that enables you to run your cloud-native applications across multiple Kubernetes clusters and clouds.
In this presentation, the maintainer of the Karmada project will share:
- A Brief introduction to Karmada.
- New features over the last year
- Application Priority Scheduling
- Federated ResourceQuota Enforcement
- Stateful Application Cluster Failover
- AI Jobs Scheduling Enhancements
- Remarkable Performance Optimization
- Karmada Dashboard Release
- Karmada Operator Enhancement
- Real-world case studies
- Overview of the community
- Roadmap
- QA
Flink on Karmada: Building Resilient Data Pipelines on Multi-Cluster K8s
Karmada is an increasingly popular open source tool for deploying and managing cloud-native applications across Kubernetes clusters. It can also be used to boost workload resiliency with its existing failover support. But what happens if we need to conserve state?
Within the context of data processing (e.g., Apache Flink or Apache Spark), the state is often critical to making sure workloads are able to gracefully resume in the event of a disruption. In collaboration with the Karmada community, the Bloomberg Streaming Analytics team has worked to bridge this gap in Karmada’s existing failover features.
During this talk, we’ll use a real-life Flink on Karmada use case to discuss:
- The complexities related to intelligently scheduling stateful workloads, improving resiliency, and ensuring state consistency during failover on multi-cluster K8s
- The open source enhancements to Karmada to manage these challenges
- How to leverage Karmada to support other stateful use cases!
How Bloomberg Creates a Resilient Data Analytics Platform Using Karmada
Bloomberg’s Data Analytics Platform Engineering team supports a wide-range of real-time streaming, large batch ETL, and data exploration use-cases by using Apache Flink, Apache Spark, and Trino across multi-cluster Kubernetes. However, deploying and managing these workflows at scale efficiently can be challenging due to varying resource requirements and uptime needs. For stateful applications like Apache Flink, ensuring recovery and state conservation after downtime is especially important.
This session will discuss how Bloomberg uses Karmada, a multi-cluster management system, to deploy and manage Apache Flink. We’ll also explore how Karmada’s capabilities can be expanded to handle additional data analytics workloads, including Apache Spark and Trino. The session will cover the unique requirements and real-life use-cases for each, including:
- Resource-aware workload scheduling
- Custom resource requirements and health interpretation
- State conservation during application failover
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top