Azure Data Platform
Birmingham, United Kingdom
Paul is a Microsoft Data Platform MVP with 10+ years’ experience working with the complete on premises SQL Server stack in a variety of roles and industries. Now as Data Analytics Consultant has turned his keyboard to big data solutions in the Microsoft cloud. Specialising in Azure Data Lake Analytics, Azure Data Factory, Azure Stream Analytics, Event Hubs and IoT. Paul is also a STEM Ambassador for the networking education in schools’ programme, PASS chapter leader for the Microsoft Data Platform Group – Birmingham, a member of the SQL Relay committee, SQL Bits, SQL Saturday, PASS Summit speaker and helper. Currently the Stack Overflow top user for Azure Data Factory. As well as very active member of the technical community.
Regular blood donor, father to daughter Neave, husband to Christine and all round good egg! :-)
Microsoft’s Cognitive Services are basically the best thing since sliced bread, especially for anybody working with data. Artificial intelligence just got packaged and made available for the masses to download. In this short talk, I’ll take you on a whirl wind tour of how to use these massively powerful libraries directly in Azure Data Lake with that offspring of T-SQL and C# ... U-SQL. How do you get hold of the DLL’s and how can you wire them up for yourself?... Everything will be revealed as well as the chance to see what the machines make of the audience!
The desire and expectation to use real-time data is constantly growing, businesses need to react to market trends instantly. In this new data driven age a daily ETL load/processing window isn’t enough. We need a constant stream of information and analytics achieved in real-time. In this session will look at how that can be achieved using Azure Stream Analytics. Building streaming jobs that can blend and aggregate data as it arrives to drive live Power BI dashboards. Plus, we’ll explore how a complete lambda architecture can be created when combining stream and batch data together.
What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a feature rich, graphical, scalable data flow environment to rival any ETL tech we’ve previously had available in Azure. In this session we’ll look at Azure Data Factory v2 and how it integrates with Azure Data Bricks to produce a powerful abstraction over the Apache Spark analytics ecosystem. Now we can transform data in Azure using Data Bricks but without the need to write a single line of Scala or Python! If you haven’t used either service yet, don’t worry, you’ll get a quick introduction to both before we go deeper into the new ADF Data Flow feature.
If you have already mastered the basics of Azure Data Factory (ADF) and are now looking to advance your knowledge of the tool this is the session for you. Yes, Data Factory can handle the orchestration of our ETL pipelines. But what about our wider Azure environment? In this session we’ll go beyond the basics looking at how we build custom activities, metadata driven dynamic design patterns for Data Factory. Plus, considerations for optimising compute costs by controlling other service scaling as part of normal data processing. Once we can hit a REST API with an ADF web activity anything is possible, extending our Data Factory and orchestrating everything.
SQL Server Integration Services has been a good friend since its first appearance in SQL Server 2005. But now, after a slightly bumpy start, Azure Data Factory is here and ready to replace all our DTSX package capabilities. This cloud native orchestration tool is a powerful equivalent for SSIS and the SQL Agent as a primary component within the Modern Data Warehouse. In this session we will start with the basics of Azure Data Factory. What do we need to build cloud ETL pipelines? What’s the integration runtime? Do we have an SSIS equivalent cloud data flow engine? Can we easily lift and shift existing SSIS packages into the cloud? The answers to all these questions and more in this session.
The world of data is moving quickly and traditional relational database technology can be a limiting factor in responding to change. Teams want to move quicker, work with a wider array of data, handle massive datasets and augment their code with open-source libraries and projects. Data delivery and demand for immediate insights mean we no longer have the luxury to extract, transform and load datasets before we need to realise the value locked within our data. SQL Server 2019 has made radical architecture changes to meet these challenges, introducing in-built data lakes, spark clusters, massive data ingestion engines and the ability to harness massively parallel processing architectures. These engines are all implemented behind a single, scalable interface that streamlines data acquisition, transparently and without costly movement operations.
In this talk we will outline the problems that can be tackled with the new SQL Server Big Data Clusters, provide an overview of how they have been implemented and discuss how SQL Server can now handle your Big Data problems. We will be drawing parallels to the Azure Data Platform and highlight where we can adopt similar patterns in our on-premises data platforms.
21 Apr 2018 - 21 Apr 2018
Birmingham, United Kingdom