Paul Andrew

Information & Communications Technology

Azure Data Platform

Derby, England, United Kingdom

Azure Data Factory - What? Why? How? Let's Build It!

If we want to achieve any data processing in Azure we need an umbrella service to manage, monitor and schedule our solution. For a long time when working on premises, the SQL Agent has been our go-to tool, combined with T-SQL and SSIS packages. It’s now time to upgrade our skills and start using cloud native services to achieve the same thing on the Microsoft Cloud Platform. Within a PaaS only Modern Data Warehouse, the primary component for delivering that orchestration is Azure Data Factory, combined with Azure Databricks.

In this full day of training we’ll start with the basics and then get hands on with the tools. We’ll build our own Azure ETL/ELT pipelines using all Data Factory has to offer. Plus, consider hybrid architectures, think about lifting and shifting legacy packages, and explore complex bootstrapping to orchestrate everything in Azure with Data Factory.

Specifically, we’ll be covering the following topics:

- An introduction to Azure Data Factory. What is it and why use it?
- How to extend our orchestration processes with Custom Activities, Azure Functions and Web Hooks.
- Using SSIS packages in Azure.
- Data Factory Data Flows (Mapping & Wrangling) with support from Azure Databricks
- Dynamic metadata driven pipelines.
- Data Factory alerting and monitoring.
- Data Factory DevOps.
- Lessons learnt from using Azure Data Factory in production.

Azure Data Factory has been around since 2015 and has matured massively to form a complete feature rich cloud offering. Add this tech to your development toolbox today! Join me for this complete lesson in everything you need to deliver Azure Data Factory within your data platform solution.

Have your laptops to hand and come armed with your Azure subscription, including credit and rights to deploy resources.

Please check with bill payer.

Infrastructure not included.


Paul Andrew

Principal Consultant & Solution Architect, Data Platform MVP

Principal consultant and architect specialising in big data solutions on the Microsoft Azure cloud platform.
Data engineering competencies include Azure Data Factory, Data Lake, Databricks, Stream Analytics, Event Hub, IoT Hub, Functions, Automation, Logic Apps and of course the complete SQL Server business intelligence stack.
Many years’ experience working within healthcare, retail and gaming verticals delivering analytics using industry leading methods and technical design patterns.
STEM ambassador and very active member of the data platform community delivering training and technical sessions at conferences both nationally and internationally.
Father, husband, swimmer, cyclist, runner, blood donor, geek, Lego and Star Wars fan!

Paul's full speaker profile