Krzysztof Zarzycki
CTO @ Getindata | Xebia Data
Zürich, Switzerland
Actions
Krzysztof is the co-founder and CTO of GetInData | Part of Xebia. Armed with a major in Computer Science from Warsaw University, he has spent 15 years paving the way in the field of software, big data and AI. Krzysztof has worked in a range of roles, including engineer, to architect and consultant, which have led him to current position as CTO. Krzysztof is passionate about the latest innovations in the data world, especially obsessed with advances around real-time processing, MLOps and AI brining the real business value.
Area of Expertise
Topics
Building a Scalable SQL-Based Streaming Analytics Platform with Apache Flink, Kafka, dbt
Discover how our Streaming Analytics Platform, powered by Apache Flink, Kafka, dbt and more, enables organizations to scale their real-time capabilities. With a focus on simplicity and efficiency, the platform leverages SQL for defining workloads. What’s more, we integrated dbt with Flink SQL to bring the industry’s best engineering practices to building streaming data pipelines.
We have developed a framework that facilitates effortless transition from local development to production on multiple available Flink platforms.
To enhance accessibility, we have built a cloud workbench based on JupyterHub and VSCode that allows for for data exploration, analysis, and development of streaming applications.
We proudly open sourced our tools, inviting everyone to develop them together.
Join us to explore the architecture, the open-source tools we developed, the benefits, and real-world use cases of this platform.
dbt run SQL-based real-time streaming pipelines on Apache Flink
In this presentation, we will explore the dbt-Flink adapter, a project we developed at GetInData and we open-sourced. On one one side brings the power of dbt's best data engineering standards to Apache Flink and its community. On the other , it enables dbt users to run their SQL-based data transformations in streaming, real-time. As cherry on the top we will show how Flink with dbt can be used as a generic real-time and batch end-to-end ETL system.
Our focus will be on the technical aspects and practical applications of this integration. We will delve into the details of how the adapter works, its benefits, and real-world use cases. Join us to learn how to leverage Apache Flink and streaming in your existing dbt projects, and how to raise the quality of your SQL-based Flink pipelines with dbt.
CEP platform handling millions of users - lessons from 3 years in production
3 years ago together with the leading telco in Kazakhstan - Kcell, we started a project of building Flink-based Complex Event Processing (CEP) platform for their business. The system built handles now over 10 millions of users, 500 K of interactions per second and billions per day. On these massive streams of interactions it applies in real-time over a hundred of complex pattern searches, analytics and business rules e.g. to offer new products, next best actions, communicate to users, detect frauds or automate various business actions. We with Kcell jointly run and improve this platform for 3 years till today while the use cases, their complexity and data sizes are only expanding. In our presentation we show how we architected and built the platform, what advanced and innovative Flink patterns we applied like dynamic updates of application logic or running tens of rules in one application. We also share how we evolve the system, operate it and make sure it runs 24/7. We won’t skimp on the lessons we learned from running, sometimes failing, but also optimizing our applications for high load and performance. We also show what value the project brings to the client and how it enabled thinking about new products and revenue streams. Whether you're in a telecom, banking, e-commerce, IoT or other data intensive industry, you will learn a ton with us.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top