![Kamil Nowinski](https://sessionize.com/image/4fcb-400o400o2-ca-cf6c-4e53-b198-e39cbde2aa4c.058baf92-9645-4d59-b808-0dda979beea5.jpg)
Kamil Nowinski
Blogger, Speaker, Microsoft Data Platform MVP. Group Manager & Analytics Architect. MCSE Data Management and Analytics
Bloger, prelegent, członek #sqlfamily. Microsoft Data Platform MVP, friend of Redgate. Pasjonat danych, architekt oraz inżynier danych.
Stevenage, United Kingdom
Blogger, speaker, #sqlfamily member. Microsoft Data Platform MVP. Data passionate, Data Engineer and Architect.
Over 20 years of programming and experience with SQL Server databases (since 2000 version) he confirmed by certificates MCITP, MCP, MCTS, MCSA, MCSE Data Platform & Data management & analytics. He worked both as developer and administrator big databases designing systems from the scratch. Recently focused on Data Platform in Azure as a certified (Azure Dev-Ops Engineer Expert, Azure Developer Associate) Data Engineer and Azure Architect.
Passionate about optimization of database systems, an advocate of code transparency, open-source projects and automation, DevOps and PowerShell fan.
Since 2015 he has been living and working in the UK. Currently professionally associated with Avanade, an international consulting company.
Socially, tied with Data Community Poland (former PLSSUG) for many years, He worked a couple of years as a volunteer and now as a co-organizer and speaker of the biggest SQL Server conference in Poland (SQLDay).
Socially, tied with the Data Community Poland Association (former PLSSUG) and the SQLDay conference for many years; currently as a co-organizer of a five-day Data Relay conference in the UK, a volunteer at SQLBits and a presenter at these and many other conferences.
An originator of the "Ask SQL Family" podcast and founder of SQLPlayer blog.
Privately happy husband and father of two wonderful girls.
Bloger, prelegent, członek #sqlfamily. Microsoft Data Platform MVP, friend of Redgate.
Pasjonat danych, architekt oraz inżynier danych.
Ponad 20 lat doświadczenia z bazami danych SQL Server (od wersji 2000) potwierdzone wieloma certyfikatami (MCITP, MCP, MCTS, MCSA, MCSE Data Platform oraz Data Management & Analytics). Umiejętności zdobywał zarówno jako programista jak i administrator dużych baz danych projektując systemy od podstaw.
Pasjonat optymalizacji systemów bazodanowych, zwolennik przejrzystości kodu, projektów open-source oraz automatyzacji, fan DevOps i PowerShell'a.
Od 2015r mieszka i pracuje w UK. Obecnie związany zawodowo z Avanade, firmą konsultingową o międzynarodowym zasięgu.
Społecznościowo przez wiele lat związany ze Stowarzyszeniem Data Community Poland (dawny PLSSUG) oraz konferencją SQLDay; obecnie jako współorganizator pięciodniowej konferencji Data Relay w UK, wolontariusz na SQLBits oraz prezenter na tych i wielu innych konferencjach.
Pomysłodawca podcastu "Ask SQL Family" oraz założyciel bloga SQLPlayer.
Prywatnie szczęśliwy mąż oraz ojciec dwóch wspaniałych dziewczyn.
Links
Area of Expertise
Topics
Azure DevOps from the field
In the past Azure DevOps (or VSTS actually) was not that great tool when it comes to managing projects, collaboration,
not mentioning area of deployment. In result many people had to complement it with tools like TeamCity, Octopus, etc.
This changed.
For many years, Azure DevOps is very mature, versatile and enough for most Azure projects.
During this workshop, I want to share my practical experience from the field, from practical projects across several customers I worked with.
We will go through the portal from A to Z, learning purpose of each module: Overview, dashboard, Wiki, Boards (features, user stories, tasks, sprints).
I will explain how to start working in organisation, what's the scope of organisation, project, Teams and (what's even more important) how to organise code in GIT repository, when building data platform (as an example), splitting code and pipelines into Infra and App pieces.
I will touch topics like ARM Template, BICEP, Terraform, explain CI/CD and show how to prepare build and release pipelines.
We will see the most popular tasks and approaches to achieve a goal with minimal efforts having some templates and working examples.
Azure Data Factory - Deployment challenges
ADF is an important brick in the architecture of any modern data warehousing solution and many other scenarios.
As it exists for some time now and we know its capability pretty well, the deployment of the service is still something that leaves much to be desired, specifically in a bit more complex instances.
In this session, I will show a few challenges to publishing ADF and solution for them.
ADF Deployments with Azure DevOps
Azure Data Factory is a great orchestration tool in the cloud, is mature and for a while now with us.
Authoring the pipelines and other objects as a developer via browser (v2), working appropriately with branches, debug mode and understanding an integration with Git repo might be a bit tricky.
If you add to this, the need for deployment to different environments, adf_publish branch and why actually two methods of deployment exist - these things can be overwhelming.
Learn the best ways of working with ADF, scripts and tools for deployment and differences between them. See, how to automatically (not via UI) generate/export arm template files and use them in further steps in Azure DevOps, if you prefer using this way.
Delta Lake Tables 101
There are more and more file formats nowadays: Parquet format is not the best shiny star any longer. Now, the Delta Lake takes the prim.
Why people do confuse it with Parquet and always talk about files in this case?
In this session, we'll take a look at the evolution of ETL into ELT and its storage aspect, which explain why it is "a must" for modern data warehouse solutions and how is it related Delta Lake technology in cloud environments like Databricks or Synapse Analytics. Finally, we'll check what's Delta-Parquet creature presented in Microsoft Fabric OneLake recently.
We will see also what data layers (stages) are commonly set up and why they make sense.
In this session, we'll take a look at the evolution of ETL into ELT and its storage aspect, which explain why it is "a must" for modern data warehouse solutions and how is it related Delta Lake technology in cloud environments like Databricks, Synapse Analytics or the newest Microsoft Fabric.
SDLC for databases as a part of nowadays DevOps (pre-conf)
Nowadays the DevOps is the topic number one in many industries and companies. However, in some case, you see that the code repository is or will be the most important topic. The number of tasks could be overwhelming at first glance, however, there is no other way - you have to use the new tools and solutions
The code repository term is not really new but the way it is integrated with the database world is still sometimes questioned. And there are a lot more than that just naming Continuous integration, Continuous Delivery or Continuous Deployment.
I would like to show you tools to efficiently manage database projects (SQL Server Data Tools), how to start working with projects, how to deal with problems you will probably see during daily operations and how to configure and manage the projects' deployment process on different environments.
We will go to the Software Development Life Cycle (SDLC) process in great details from the database point of view. But we will not spend too much time on analysis but rather on the development part. I would like to show the usage of the Azure DevOps application.
Of course, there will be an entire module about best practices and how to efficiently use them in the database projects.
In the end, I would like to touch the cloud and show how to migrate the existing on-premises database to the Microsoft Azure SQL Database and how no to get into troubles.
After attending the workshop you will be able to do & know:
* create empty or import existing databases
* resolve various problems during the import of database
* manage database project and its objects
* handling CLR objects
* store data in a project (static data, master, etc.)
* what should be a part of a project and what shouldn't (Linked Servers, security)
* where and how to keep SQL Jobs
* split database project into more chunks and why it's required sometimes
* cope with unlimited amount of projects
* Avoid known issues like: temp tables, triggers, circular references, OPENQUERY, lack of validation
* Migrate project to Microsoft Azure (Cloud!)
* Use a hybrid approach
* Apply tSQLt Unit Tests
* Make a deployment manually and automatically (Azure DevOps)
* Distinguish (finally) all three types of "Continuous"
* some helpful PowerShell scripts
We are going to go through commercial tools as well as some tips & tricks.
Prerequisites:
Basic knowledge about: TSQL, Visual Studio & Azure DevOps;
The workshop will be done in the Visual Studio 2022 with the newest SSDT installed but you can use older version of the Visual Studio as well.
You can take your laptop with you as some of the task I'm going to do with you!
You will have access to all codes and slide deck.
Each participant of the workshop will get full, lifetime access to my "Database projects with SQL Server Data Tools (SSDT)" course worth $195 on my platform "Learn with SQLPlayer".
Databases with SSDT: Deployment in CI/CD process with Azure DevOps
When working on a database in SSDT, there is a need to deploy our changes to further environments and at the same time maintain the consistency of databases between environments. During the session, I will present how we can publish the solution manually and then go to the Continuous Integration and Continuous Deployment process using the Azure DevOps environment (formerly VSTS). In addition, we will work on inserting the unit tests, approval steps and the others using Pester and PowerShell in order to gain full automation in our database deployment process.
SSDT allows you to import and maintenance a database project within Visual Studio. Prepare a few steps more to test and deploy changes and data into target SQL Server with Azure DevOps pipelines.
SQLBits 2022 Sessionize Event
#DataWeekender v3.1 Sessionize Event
Global Azure 2021 Sessionize Event
dataMinds Connect 2020 (Virtual Edition) Sessionize Event
Data Relay 2019 Sessionize Event
Data in Devon 2019 Sessionize Event
![](https://sessionize.com/image/4fcb-400o400o2-ca-cf6c-4e53-b198-e39cbde2aa4c.058baf92-9645-4d59-b808-0dda979beea5.jpg)
Kamil Nowinski
Blogger, Speaker, Microsoft Data Platform MVP. Group Manager & Analytics Architect. MCSE Data Management and Analytics
Stevenage, United Kingdom