Session

Harnessing Large Language Models in Enterprise Data Engineering: An On-Call Revolution

Data engineering teams encounter challenges like data quality issues and pipeline failures, especially in enterprise environments. Addressing this, our approach combines the linguistic prowess of models like GPT-4 with data engineering tasks. We autonomously identify and rectify data quality issues, transform anomaly detection paradigms, and automate recovery tasks. Our methodology achieves reduced resolution times, fine-tuned anomaly detectors, and minimized downtime.

Key Takeaways:
Innovative Use of GPT-4: Leveraging large language models like GPT-4 can revolutionize traditional data engineering tasks and offer autonomous solutions.
Efficient Issue Resolution: The approach significantly reduces resolution times, allowing engineers to focus on intricate challenges.
Empirical Validation: Our case studies validate improved metrics and overall system stability, suggesting tangible benefits of integrating AI in on-call data engineering.

Mitesh Mangaonkar

Tech Lead Data Engineer at Airbnb

Seattle, Washington, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top