Arun Kumar
Staff Engineer at LinkedIn | Platform Engineering & AI Agent Infrastructure
San Francisco, California, United States
Actions
Arun is a seasoned platform and infrastructure engineer at LinkedIn. He leads large-scale platforms serving the whole engineering org, including AI agents for autonomous debugging and operations. At Ciena, he containerized monolithic applications; at Aricent, he built and scaled IoT solutions for smart metering.
Outside of work, Arun enjoys hiking, playing cricket, and spending quality time with family.
Links
Area of Expertise
Topics
Linkedin's Journey on scaling airflow
Last year, we shared how LinkedIn's continuous deployment platform (LCD) leveraged Apache Airflow to streamline and automate deployment workflows. LCD is the deployment platform inside Linkedin which is actively used by all engineers (10000+) at Likedin.
This year, we take a deeper dive into the challenges, solutions, and engineering innovations that helped us scale Airflow to support thousands of concurrent tasks while maintaining usability and reliability.
Key Takeaways:
Abstracting Airflow for a Better User Experience – How we designed a system where users could define and update their workflows without directly interacting with Airflow.
Scaling to 10,000+ Concurrent Tasks – The architectural and configuration changes that enabled us to scale execution efficiently.
Enhanced Observability & Monitoring – The tools and techniques we implemented to track Airflow’s health, detect failures, and improve reliability.
Lessons from the Field – Key learnings, trade-offs, and best practices for managing large-scale Airflow deployments.
Arun Kumar
Staff Engineer at LinkedIn | Platform Engineering & AI Agent Infrastructure
San Francisco, California, United States
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top