Speaker

Kamil Nowinski

Kamil Nowinski

Blogger, Speaker, Microsoft Data Platform MVP. Group Manager & Analytics Architect. MCSE Data Management and Analytics

Bloger, prelegent, członek #sqlfamily. Microsoft Data Platform MVP, friend of Redgate. Pasjonat danych, architekt oraz inżynier danych.

London, United Kingdom

Blogger, speaker, #sqlfamily member. Microsoft Data Platform MVP. Data passionate, Data Engineer and Architect.
Over 20 years of programming and experience with SQL Server databases (since 2000 version) he confirmed by certificates MCITP, MCP, MCTS, MCSA, MCSE Data Platform & Data management & analytics. He worked both as developer and administrator big databases designing systems from the scratch. Recently focused on Data Platform in Azure as a certified (Azure Dev-Ops Engineer Expert, Azure Developer Associate) Data Engineer and Azure Architect.
Passionate about optimization of database systems, an advocate of code transparency, open-source projects and automation, DevOps and PowerShell fan.

Since 2015 he has been living and working in the UK. Currently professionally associated with Avanade, an international consulting company.

Socially, tied with Data Community Poland (former PLSSUG) for many years, He worked a couple of years as a volunteer and now as a co-organizer and speaker of the biggest SQL Server conference in Poland (SQLDay).
Socially, tied with the Data Community Poland Association (former PLSSUG) and the SQLDay conference for many years; currently as a co-organizer of a five-day Data Relay conference in the UK, a volunteer at SQLBits and a presenter at these and many other conferences.

An originator of the "Ask SQL Family" podcast and founder of SQLPlayer blog.
Privately happy husband and father of two wonderful girls.

Bloger, prelegent, członek #sqlfamily. Microsoft Data Platform MVP, friend of Redgate.
Pasjonat danych, architekt oraz inżynier danych.
Ponad 20 lat doświadczenia z bazami danych SQL Server (od wersji 2000) potwierdzone wieloma certyfikatami (MCITP, MCP, MCTS, MCSA, MCSE Data Platform oraz Data Management & Analytics). Umiejętności zdobywał zarówno jako programista jak i administrator dużych baz danych projektując systemy od podstaw.
Pasjonat optymalizacji systemów bazodanowych, zwolennik przejrzystości kodu, projektów open-source oraz automatyzacji, fan DevOps i PowerShell'a.

Od 2015r mieszka i pracuje w UK. Obecnie związany zawodowo z Avanade, firmą konsultingową o międzynarodowym zasięgu.

Społecznościowo przez wiele lat związany ze Stowarzyszeniem Data Community Poland (dawny PLSSUG) oraz konferencją SQLDay; obecnie jako współorganizator pięciodniowej konferencji Data Relay w UK, wolontariusz na SQLBits oraz prezenter na tych i wielu innych konferencjach.

Pomysłodawca podcastu "Ask SQL Family" oraz założyciel bloga SQLPlayer.
Prywatnie szczęśliwy mąż oraz ojciec dwóch wspaniałych dziewczyn.

Awards

Area of Expertise

  • Information & Communications Technology

Topics

  • SQL Server Integration Services
  • SQL Server Data Tools
  • Azure Data Factory
  • Azure SQL DW
  • DevOps & Automation
  • Microsoft Azure DevOps
  • Azure Synapse
  • Microsoft SQL Server
  • PowerShell

Azure DevOps from the field

In the past Azure DevOps (or VSTS actually) was not that great tool when it comes to managing projects, collaboration,
not mentioning area of deployment. In result many people had to complement it with tools like TeamCity, Octopus, etc.
This changed.
For many years, Azure DevOps is very mature, versatile and enough for most Azure projects.
During this workshop, I want to share my practical experience from the field, from practical projects across several customers I worked with.
We will go through the portal from A to Z, learning purpose of each module: Overview, dashboard, Wiki, Boards (features, user stories, tasks, sprints).
I will explain how to start working in organisation, what's the scope of organisation, project, Teams and (what's even more important) how to organise code in GIT repository, when building data platform (as an example), splitting code and pipelines into Infra and App pieces.
I will touch topics like ARM Template, BICEP, Terraform, explain CI/CD and show how to prepare build and release pipelines.
We will see the most popular tasks and approaches to achieve a goal with minimal efforts having some templates and working examples.

Azure Data Factory - Deployment challenges

ADF is an important brick in the architecture of any modern data warehousing solution and many other scenarios.
As it exists for some time now and we know its capability pretty well, the deployment of the service is still something that leaves much to be desired, specifically in a bit more complex instances.
In this session, I will show a few challenges to publishing ADF and solution for them.

ADF Deployments with Azure DevOps

Azure Data Factory is a great orchestration tool in the cloud, is mature and for a while now with us.
Authoring the pipelines and other objects as a developer via browser (v2), working appropriately with branches, debug mode and understanding an integration with Git repo might be a bit tricky.
If you add to this, the need for deployment to different environments, adf_publish branch and why actually two methods of deployment exist - these things can be overwhelming.
Learn the best ways of working with ADF, scripts and tools for deployment and differences between them. See, how to automatically (not via UI) generate/export arm template files and use them in further steps in Azure DevOps, if you prefer using this way.

Azure Synapse - Deployment challenges

Azure Synapse Analytics contains various services which bring together data integration, enterprise data warehousing, and big data analytics. Such comprehensive workspace offers huge analytical capabilities making the CI/CD process very complicated simultaneously.
Automated deployment of Synapse workspace must taken a few factors into consideration: Infrastructure part of the service (pools, networking, storages) as well as application type of objects.
How to organise these things? Which tool & approach to use? What Microsoft does offer in this area?
I will touch base all these topics during the session.

Automated deployment of Synapse workspace must taken a few factors into consideration.
How to organise CI, CD, infrastructure and app part of the workspace?
I will touch base tools, approach and what Microsoft offers in this area.

Delta Lake Tables 101

There are more and more file formats nowadays: Parquet format is not the best shiny star any longer. Now, the Delta Lake takes the prim.
Why people do confuse it with Parquet and always talk about files in this case?
In this session, we'll take a look at the evolution of ETL into ELT and its storage aspect, which explain why it is "a must" for modern data warehouse solutions and how is it related Delta Lake technology in cloud environments like Databricks or Synapse Analytics. We will see what data layers (stages) are commonly set up and why they make sense.

In this session, we'll take a look at the evolution of ETL into ELT and its storage aspect, which explain why it is "a must" for modern data warehouse solutions and how is it related Delta Lake technology in cloud environments like Databricks or Synapse Analytics.

Azure Data Factory v2 with Data Flows capabilities

Microsoft's services in Azure helps us to leverage big data more easily and even more often accessible for non-technical users. Having UI in ADF version 2 - Microsoft added a new feature: Data Flow which resembles components of SSIS. This is a very user-friendly and non-code approach tool-set.
But, has that been only UI introduction? Why and how Databricks does work under the hood?
Do you want to know this new (still in private preview) feature of ADF and reveal the power of modern big data processes without knowledge of such languages like Python or Scala?
We will review this new feature of ADFv2, do deep dive to understand the mentioned techniques, compare them to SSIS and/or T-SQL and learn how modelled data flow runs Scala behind the scenes.

SDLC for databases as a part of nowadays DevOps (pre-conf)

Nowadays the DevOps is the topic number one in many industries and companies. However, in some case, you see that the code repository is or will be the most important topic. The number of tasks could be overwhelming at first glance, however, there is no other way - you have to use the new tools and solutions
The code repository term is not really new but the way it is integrated with the database world is still sometimes questioned. And there are a lot more than that just naming Continuous integration, Continuous Delivery or Continuous Deployment.
I would like to show you tools to efficiently manage database projects (SQL Server Data Tools), how to start working with projects, how to deal with problems you will probably see during daily operations and how to configure and manage the projects' deployment process on different environments.

We will go to the Software Development Life Cycle (SDLC) process in great details from the database point of view. But we will not spend too much time on analysis but rather on the development part. I would like to show the usage of the Azure DevOps application.
Of course, there will be an entire module about best practices and how to efficiently use them in the database projects.
In the end, I would like to touch the cloud and show how to migrate the existing on-premises database to the Microsoft Azure SQL Database and how no to get into troubles.
After attending the workshop you will be able to do & know:
* create empty or import existing databases
* resolve various problems during the import of database
* manage database project and its objects
* handling CLR objects
* store data in a project (static data, master, etc.)
* what should be a part of a project and what shouldn't (Linked Servers, security)
* where and how to keep SQL Jobs
* split database project into more chunks and why it's required sometimes
* cope with unlimited amount of projects
* Avoid known issues like: temp tables, triggers, circular references, OPENQUERY, lack of validation
* Migrate project to Microsoft Azure (Cloud!)
* Use a hybrid approach
* Apply tSQLt Unit Tests
* Make a deployment manually and automatically (Azure DevOps)
* Distinguish (finally) all three types of "Continuous"
* some helpful PowerShell scripts

We are going to go through commercial tools as well as some tips & tricks.

Prerequisites:
Basic knowledge about: TSQL, Visual Studio & Azure DevOps;

The workshop will be done in the Visual Studio 2022 with the newest SSDT installed but you can use older version of the Visual Studio as well.
You can take your laptop with you as some of the task I'm going to do with you!
You will have access to all codes and slide deck.

Each participant of the workshop will get full, lifetime access to my "Database projects with SQL Server Data Tools (SSDT)" course worth $195 on my platform "Learn with SQLPlayer".

Databases with SSDT: Deployment in CI/CD process with Azure DevOps

When working on a database in SSDT, there is a need to deploy our changes to further environments and at the same time maintain the consistency of databases between environments. During the session, I will present how we can publish the solution manually and then go to the Continuous Integration and Continuous Deployment process using the Azure DevOps environment (formerly VSTS). In addition, we will work on inserting the unit tests, approval steps and the others using Pester and PowerShell in order to gain full automation in our database deployment process.

SSDT allows you to import and maintenance a database project within Visual Studio. Prepare a few steps more to test and deploy changes and data into target SQL Server with Azure DevOps pipelines.

Azure Databricks 101

Many sources? Various format? Unstructured data? Big Data? You might think that these only a buzz words. Not really. These days it's a part of modern data flow architecture. No matter what do you use - SQL Server, Cosmos DB, Azure SQL DW, Azure Data Factory, Data Lake... somewhere there you can find Databricks. So, the question is: what the Azure Databricks is and which scenario it could be used in?
Use Databricks to analyse large DataSets at scale, write Python, Scala or SQL command in one notebook to ingest, process and push the data to the required target. Use Databricks' notebook as a part of Azure Data Factory pipeline. We also will try to answer whether Databricks would replace SSIS as a modern ETL/ELT process?
If you are wondering about all these things - you should join me in this session.

Azure Databricks for the beginners where we will try to understand in which scenarios the notebooks and Spark cluster can be leverage and helpful.

Lightning Talk: Reference/master data for database project

By default, SSDT (SQL Server Data Tools) does not offer capabilities for deploying data of server-level objects. In this talk, I will show you how quickly fill that gap and generate a script with INSERT/MERGE statements in it.

Lightning Talk: Cosmos DB - when yes and when not?

Azure Cosmos DB offers single-digit-millisecond data access to NoSQL database. But what that does mean and when exactly we should use it? We will go through a few scenarios where Cosmos DB suits very well and an example when completely doesn't.

Azure Cosmos DB introduction

Cosmos DB is Microsoft's globally distributed, multi-model database service. It's a database from NoSQL family, but it does not mean that SQL is not engaged in there. During the session, I will explain what is the service, how many different APIs we can use, how elastically you can scale throughput and storage and what kind of scenarios are good to go with this technology. The demo shows you how to start with Cosmos DB and what kind of things you should be aware of.

SQLBits 2022

March 2022 London, United Kingdom

Global Azure 2021

April 2021

SQLDay 2020

November 2020 Wrocław, Poland

dataMinds Connect 2020 (Virtual Edition)

October 2020 Mechelen, Belgium

SQLBits 2020

September 2020 London, United Kingdom

SQLSaturday Slovenia

December 2019 Ljubljana, Slovenia

SQL Saturday #926 Lisbon

November 2019 Lisbon, Portugal

Data Relay 2019

October 2019

SQL Saturday #904 Madrid

September 2019 Madrid, Spain

SQL Saturday #898 Gothenburg

September 2019 Göteborg, Sweden

SQL Saturday #857 Kyiv

May 2019 Kyiv, Ukraine

SQLDay 2019

May 2019 Wrocław, Poland

Data in Devon 2019

April 2019 Exeter, United Kingdom

SQLBits 2019

February 2019 Manchester, United Kingdom

SQL Saturday #829 Pordenone

February 2019 Pordenone, Italy

Kamil Nowinski

Blogger, Speaker, Microsoft Data Platform MVP. Group Manager & Analytics Architect. MCSE Data Management and Analytics

London, United Kingdom