Srijan Saket
Staff Machine Learning Engineer at ShareChat, transforming India's content ecosystem by building largest social media platform for Bharat
Seattle, Washington, United States
Actions
Srijan Saket currently works on developing scalable and cost-effective recommender systems. As a pioneering member of the AI team, he played a pivotal role in establishing the framework for machine learning at ShareChat. Over the last 6+ years, he worked on transitioning ML projects from research to production, handling tasks such as automated content moderation, creating recsys for new categories, & developing scalable, efficient feature pipelines for ranking models. His efforts significantly contributed to the platform's growth, expanding its user base from <1m to 200m+ strong community. His current research focuses on early stage recommendation, content journey in recommender systems, candidate retrieval for multi-objective ranking, and scalable machine learning systems. Srijan has completed his bachelors and masters from IIT Kanpur and some of his recent work has been included at top conferences including WWW, RecSys. He delivered the keynote speech (Industrial track) in the recently held FIRE 2023 conference in Goa, IN. He also has a US patent on human assisted chatbot conversations.
Area of Expertise
Topics
Real-time Event Joining in Practice With Kafka and Flink
Historically, machine learning training pipelines have been heavily utilizing batch training models, i.e., getting retrained every few hours. However, industrial practitioners have proved that real-time training can yield a more adaptive and personalized user experience. The transition from batch to real-time is full of tradeoffs to get the benefits of accuracy and freshness while keeping the costs low and having a predictable, maintainable system.
This session will delve deeper into our journey of migrating to a streaming pipeline for our ML models using Kafka and Flink. You will learn how to transition from Pub/Sub to Kafka for incoming real-time events and leverage Flink for streaming joins using RocksDB and checkpointing. We will also discuss navigating nuances like causal dependency between events, event-time versus processing time, and exactly-once vs atleast-once delivery, among others.
Furthermore, you will see how we utilized topic partitioning in Kafka to improve scalability, reduced the throughput of events by 85% using Avro schema and compression, decreased cost by 40%, and set up a separate pipeline to ensure correctness.
Attending this session, you'll gain a deeper understanding of the tradeoffs and nuances in real-time systems, allowing you to make well-informed decisions that suit your specific requirements.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top