Tzu-Li (Gordon) Tai
Apache Flink PMC, Staff Software Engineer at Confluent
Tzu-Li (Gordon) Tai is a Staff Software Engineer at Confluent, and a long-time Apache Flink Committer and PMC member. At Confluent, he is currently focusing on making Flink and Kafka work really well together. His past contributions in Flink spans various areas, including the Stateful Functions subproject (https://statefun.io), some of the more popular connectors for Flink (Apache Kafka, AWS Kinesis, etc.), as well as several topics surrounding evolvability of stateful Flink streaming applications.
State Unlocked
As stateful streaming processing becomes more and more mature for complex event-driven applications and real-time analytics, users have put Apache Flink into the center of their business logic and entrusted it to manage their most valuable assets, their application data, as internal state of Flink streaming pipelines.
At the same time, the Flink community has continued efforts to make sure that users feel safe and future-proof in doing that. They should have sufficient means to access and modify their state, as well as making it much easier to bootstrap state with existing data from external systems. These efforts span multiple Flink major releases and consists of the following: 1) evolvable state schema, 2) flexibility in swapping state backends, and 3) an offline tool to read, process, and create new snapshots that streaming applications can bootstrap its state with.
In this talk, we will go over these topics and demonstrate how users can interact with state with the availability of these new features.
Stateful Functions: Polyglot Event-Driven Functions for Stateful Distributed Applications
Frameworks like Kubernetes have solved most of the challenges of dealing with stateless applications. But for stateful applications, we are still clinging to the ancient wisdom that state shall be someone else's problem: just put it in a database! Because of that, we are still struggling with the same issues of data consistency and complex failure semantics as a decade ago. For bigger applications, we are still far away from the easy development and operations experience that is the promise of Serverless.
In this talk, we will do a deep dive into Apache Flink's newest addition: "Stateful Functions", an API and library on top of Flink that attempts to solve that very problem of scalable and resilient distributed applications. Users provide containerized, polyglot event-driven functions that can be scaled in a lightweight manner. A Stream Processor (Apache Flink) handles the messaging between functions and state consistency in a fault tolerant way, subsuming the role of the database.
This talk offers an in-depth walkthrough of how the Stateful Functions runtime works, as well as the building blocks that are provided to users to structure a Stateful Functions application.
Stateful Functions is currently being migrated into the Apache Flink project. At the moment, code, docs, and examples are still under https://statefun.io/.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top