Spark is the platform for compute on big data workloads in the cloud and underpins the two analytics platforms in Azure, Databricks and Microsoft Fabric. Python is the primary language for building modern data pipelines and to best leverage the capabilities of spark.

In this session we’ll walk through how best to develop in Python and notebooks across both Databricks and Microsoft Fabric, with a specific emphasis on developer workflow, performance tuning, and “how to actually build like a pro” in each environment.

We’ll cover:

• Development environment and project layout
• Modularity and reusability
• Compute optimization
• Performance tuning for Spark DataFrames
• How each platform uses GenAI assistants to speed development

This isn’t a high-level comparison, with demos in both platforms, we will get you up to speed and ready to output efficient, clean code ready for production environments.