Everyone knows data has exploded in the past ten years. Even small companies now employ teams of data scientists and analysts tasked with data exploration and analysis of a wide variety of data sources. Consequently, it is also not uncommon for bloated, long-running data pipelines to terminate in unwieldy data warehouses managed by exhausted engineering teams. The whole data system can be frustrating for data scientists and unsustainable for engineers.

When we shift our mindset away from data warehouses as static resources for data teams to ingest, and toward data warehouses as living software for data teams to actually use, data warehouses can (and should!) be delivered the DevOps way. We will discuss real world examples of how DevOps-ing our data pipelines can lead to better data scientist and analyst experiences and more efficient data engineering teams.