Ever since the rise of DevOps, the golden rule has been to separate the technology stack into DEV, TEST, and PROD environments. Over the last 10 years, data engineers and data platforms have been expected to follow the same environment split as app developers. DEV is used for development work, obviously, with synthetic or fake data. The ingestion endpoints hold almost no data, and are themselves unreliable, as system developers frequently change them. After all, we are in DEV.

For data engineers, this setup has never truly worked. Data development requires access to production-quality data, with foreign key relationships preserved across multiple systems. As a result, significant development often occurs in production or the development environment, using production data in insecure settings.

What would it look like to build a dedicated DEV environment tailored to the data development lifecycle—complete with access to production-grade data, a proper DEV/Prod split, a CI/CD pipeline, and headless PROD—without compromising security?
In this session, I describe how I have built exactly that. These environmental principles are applied in highly regulated industries with strict security requirements. Here I will share how I have structured them, using Snowflake and DBT to achieve an outcome that satisfies both data engineers and security officers.