In this session, we'll unravel the story of ShareChat's transition from Java to Benthos for crafting efficient ETL pipelines. With a single configuration file, Benthos effortlessly connects diverse sources and sinks, transforming the way we handle data. Discover how, armed with "at least once" guarantees, Benthos emerged as the go-to solution for our stateless pipelines.

Focused on real-world use cases, we'll explore how Benthos, paired seamlessly with Kafka and Kafka Streams, became the linchpin of our operations. From sending notifications to triggering events for millions of users, to routing posts for reviews, Benthos proved its mettle in simplifying complex tasks.

Learn how we platformized Benthos at ShareChat, deploying over 20 jobs from a single repository. Delve into the specifics of how Benthos with Kafka and Kafka Streams powers our data pipelines, handling tasks such as dumping data to databases, making API calls, and routing events to different message queues. The session will highlight the simplicity and performance benefits we've achieved—processing 2K events/sec to an impressive 45K events/sec—without the need for extensive developer code.