Session
Orchestrating Data Workflows with Apache Airflow in Microsoft Fabric
This session will explore the common challenges faced by users, the innovative solutions provided by Microsoft Fabric, and how to effectively implement and manage Airflow jobs for your data projects.
Key Takeaways:
1. Understand the pain points associated with traditional Airflow setups, including complexities in configuration, security management, and time inefficiencies.
2. Learn how Microsoft Fabric addresses these challenges through a managed version of Apache Airflow, focusing on instant provisioning, security integration, and intelligent resource management.
3. Discover the user-friendly Integrated Development Environment (IDE) that simplifies code deployment and management of workflows.
4. Explore the Git integration for version control and CI/CD processes, enhancing collaboration and efficiency in development.
5. Get insights into the pricing structure of managed Airflow in Fabric and how to optimize costs based on your workload needs.
6. Hear about future enhancements and features planned for Airflow in Microsoft Fabric, including improved network security and identity management.
Agenda:
- Introduction to Apache Airflow and Microsoft Fabric (10 minutes)
- Overview of Airflow and its role in data orchestration.
Introduction to Microsoft Fabric and its features.
Identifying Customer Pain Points (10 minutes)
- Discuss challenges in setting up and managing Airflow.
Highlight security concerns and update management issues.
Managed Airflow Solutions in Microsoft Fabric (15 minutes)
-instant provisioning of Airflow clusters.
Integration with Azure AD for enhanced security and compliance.
Auto-scaling features to optimize resource utilization.
Developing Workflows with the IDE (10 minutes)
- Demonstration of the user-friendly IDE for deploying Airflow jobs.
Best practices for managing workflows effectively.
Git Integration and CI/CD Processes (15 minutes)
- How to leverage Git sync for version control.
Implementing CI/CD pipelines with Airflow in Microsoft Fabric.
Pricing Overview and Cost Optimization (10 minutes)
- Breakdown of the pricing model for managed Airflow.
Tips for optimizing costs based on workload requirements.
Future Enhancements and Roadmap (10 minutes)
Discussion of planned features and improvements based on customer feedback.
Focus on network security and identity management enhancements.
Q&A Session (10 minutes)
Target Audience:
Data engineers and data scientists interested in workflow management.
IT professionals looking to implement Apache Airflow in a cloud environment.
Organizations using or considering Microsoft Fabric for data orchestration.
Prerequisites:
Basic understanding of Apache Airflow and data workflows.
Familiarity with cloud services and Microsoft Fabric is a plus.

Sally Dabbah
Empowering innovation through Azure’s boundless possibilities
Herzliya, Israel
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top