Paul Andrew

Information & Communications Technology

Azure Data Platform

Derby, England, United Kingdom

ETL in Azure Made Easy with Data Factory Data Flows (Mapping & Wrangling)

What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a feature rich, graphical, scalable data flow environment to rival any ETL tech we’ve previously had available in Azure. In this session we will look at Azure Data Factory and how it integrates with Azure Databricks to produce a powerful abstraction over the Apache Spark analytics ecosystem in the form of Mapping and Wrangling Data Flows. If you have ever transformed datasets using SQL Server Integration Services (SSIS) packages or via Power BI’s Power Query tool this is the session for you. Now we can transform data in Azure using our favourite interfaces but with the support of Azure Databricks doing the heavy lifting. In this session you will get a quick introduction to Azure Data Factory before we go deeper into the services new Mapping and Wrangling Data Flows features. Start using cloud native technology and scale out compute within a convenient, easy to use Data Factory rich graphical interface.

Paul Andrew

Principal Consultant & Solution Architect, Data Platform MVP

Principal consultant and architect specialising in big data solutions on the Microsoft Azure cloud platform.
Data engineering competencies include Azure Data Factory, Data Lake, Databricks, Stream Analytics, Event Hub, IoT Hub, Functions, Automation, Logic Apps and of course the complete SQL Server business intelligence stack.
Many years’ experience working within healthcare, retail and gaming verticals delivering analytics using industry leading methods and technical design patterns.
STEM ambassador and very active member of the data platform community delivering training and technical sessions at conferences both nationally and internationally.
Father, husband, swimmer, cyclist, runner, blood donor, geek, Lego and Star Wars fan!

Paul's full speaker profile