Azure Data Platform
Derby, England, United Kingdom
If you have already mastered the basics of Azure Data Factory (ADF) and you are now looking to advance your knowledge of the tool, this is the session for you. Yes, Data Factory can handle the orchestration of our ETL pipelines. But what about our wider Azure environment? In this session we will take a deeper dive into the service, considering how to build custom activities, create metadata driven dynamic pipelines and think about hierarchical design patterns. Plus, explore ways for optimising our Azure compute costs by controlling other resource scaling as part of our normal data processing pipelines. How? Well, once we can hit a REST API from an ADF web activity anything is possible, extending our Data Factory and orchestrating everything in any data platform solution. All this and more in a series of short lessons (based on real world experience) I will take you through how to use Azure Data Factory in production. Finally, we will look at how Data Factory can be deployed using Azure DevOps.
Principal consultant and architect specialising in big data solutions on the Microsoft Azure cloud platform.
Data engineering competencies include Azure Data Factory, Data Lake, Databricks, Stream Analytics, Event Hub, IoT Hub, Functions, Automation, Logic Apps and of course the complete SQL Server business intelligence stack.
Many years’ experience working within healthcare, retail and gaming verticals delivering analytics using industry leading methods and technical design patterns.
STEM ambassador and very active member of the data platform community delivering training and technical sessions at conferences both nationally and internationally.
Father, husband, swimmer, cyclist, runner, blood donor, geek, Lego and Star Wars fan!