Norm Warren
Data engineer, mountian biker, former break-dancer !
Salt Lake City, Utah, United States
Actions
My name is Norm Warren.
Links
Area of Expertise
Topics
Growing Data Compliance - Tools & Tips for SOC2, GDPR, CCPA
It is sad when a company must shift focus from growing to fulfilling compliance requirements !!! I will go over the tools and tips used for data compliance, an area which will continue to grow so that this "hoop" is easier.
DBT for Microsoft Fabric Warehouses
Microsoft Fabric is the latest and there are benefits of using DBT (data build tools) with it. I will review benefits of using both to create a data pipeline from source to visualization.
Data Work, Turbocharged: How AI Assist Tools Change the Game
Data work should be faster with AI tools ! This session will go over the most common tools (today), how to use those to your advantage to improve speed of development and a peak into upcoming AI developer-assist tools.
Serverless ETL with AWS Glue
Introduction
-An Overview of traditional ETL in comparison to AWS Glue
-An Overview of AWS Glue
-Demo: Creating an ETL Solution Using AWS Glue
-Some Use Cases for Using AWS Glue
-Summary
What you will gain from this course:
-An understanding of Serverless ETL which means extract, transform, and loading of data.
-Knowledge of architecture of a typical ETL project between source data and destination databases, data warehouse, or Big Data destinations.
-Understanding of prerequisite setup of AWS parts to use AWS Glue for ETL.
-Knowledge of how to use AWS Glue to perform Serverless ETL.
-How to edit ETL processes created from Glue.
Pre-requisites:
-Understanding of one or more of the data destinations offered by AWS.
-An awareness of data warehousing principles.
-Helpful: understanding of Serverless computing. See video: What Serverless Computing?
-Understanding of object-oriented programming, such as Python.
Infrastructure as Code 101 - Ansible + Terraform
Go over the why briefly and some examples where we have incorporated Terraform and Ansible to apply infrastructure as code in practice, the benefits and gotchas.
Treating Analytics as Code with dbt (Data Build Tools) 101
The modern data stack and how we structure and orchestrate our pipelines with cloud databases (BigQuery, Snowflake, Redshift, and more) has advanced and become streamlined with a relatively new tool called Data Build Tools (dbt).
There are numerous features which make easier the treating of analytics as code. See the DataOps Manifesto (https://dataopsmanifesto.org/en/) by adding CI features, automated testing, the ability to edit SQL in VS Code, and orchestrate data modeling.
Data observability in SQL Server + dbt + open-source packages: Elementary and DataFold's DataDiff
I work at a startup company and the only data engineer and consistently search for tools that deliver great value. Elementary and DataDiff are open-source tools which help answer the following questions.
Is the data up-to-date?
Is the data complete?
Are fields within expected ranges?
Is the null rate higher or lower than it should be?
Has the schema changed?
Norm Warren
Data engineer, mountian biker, former break-dancer !
Salt Lake City, Utah, United States
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top