Serjesh Sharma
Supervisor ADAS MLOps
Actions
Serjesh Sharma is seasoned data analytics and machine learning leader with rich experience in data science, machine learning, cloud services and MLOPS. He has successfully executed many machine learning projects related to NLP, CV,chatbot, ML pipeline standardization and automation, feature stores and traditional data science use cases for fortune 500 companies. He previously held roles in ML/Data engineering both as an individual contributor and tech lead at various point in his career.
Links
Airflow at Ford: A Job Router Training Advance Driver Assistance Systems
Ford Motor Company operates extensively across various nations. The Data Operations (DataOps) team for Advanced Driver Assistance Systems (ADAS) at Ford is tasked with the processing of terabyte-scale daily data from lidar, radar, and video. To manage this, the DataOps team is challenged with orchestrating diverse, compute-intensive pipelines across both on-premises infrastructure and the GCP and deal with sensitive of customer data across both environments The team is also responsible for facilitating the execution of on-demand, compute-intensive algorithms at scale through. To achieve these objectives, the team employs Astronomer/Airflow at the core of its strategic approach. This involves various deployments of Astronomer/Airflow that integrate seamlessly and securely (via Apigee) to initiate batch data processing and ML jobs on the cloud, as well as compute-intensive computer vision tasks on-premises, with essential alerting provided through the ELK stack. This presentation will delve into the architecture and strategic planning surrounding the hybrid batch router, highlighting its pivotal role in promoting rapid innovation and scalability in the development of ADAS features.
Adaptive task sizing for efficient GKE use using logs for ADAS usecases
Ford Motor Company, a leading global automotive manufacturer, maintains a significant operational presence worldwide. Within Ford, the Data Operations (DataOps) team for Advanced Driver Assistance Systems (ADAS) is entrusted with the critical role of processing petabyte-scale datasets derived from lidar, radar, and video sources. The data processed by the team is instrumental in both enhancing existing ADAS features and facilitating the development of future advancements in the realms of ML/AI.
A pivotal aspect of managing data at such a scale involves the meticulous optimization
of tasks within a High-Performance Compute (HPC) cluster to achieve optimal parallelization and efficient queuing mechanisms. To do this, the ELK stack is essential to continuously predict and adjust the size of tasks for precise calibration of our computational resources. In this talk, we will share our journey, focusing on the challenges encountered in scaling our operations and the architectural solutions we implemented to facilitate adaptive task sizing using log analysis.
Airflow Summit 2024 Sessionize Event
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top