At Pinterest, we streaming data processing team built flink as a service to onboard and operate mission critical use cases, directly affect Pinterest revenue and user engagement. In this talk, we will share the customization we made on top of open source flink to enhance reliability, development speed, and cost-effectiveness. Our discussion will feature:
Common Libraries: We will talk about our unified source offering unlimited log, essential operators like deduplication, and how we scale the service connectors.
Flink Internals: Enhancements in serialization, scheduler improvements, and mini batch optimizations will be covered to demonstrate how we've boosted our processing efficiency.
Ecosystem: Delve into our monitoring systems, job debugging, job tuning, and the streamlined procedures we use to move job from development to production seamlessly.

Join us to uncover the innovative techniques we use to advance the capabilities of our streaming processing at Pinterest.