Many transformations are fine candidates for concretizing with dbt. But there are transformations that live in the data science world that are not well-suited for dbt—and probably for good reason. Consider the total set of all transformations, from mandatory pre-processing steps to sophisticated statistical transformations (e.g., converting data types versus computing robust measures of central tendency). The question quickly becomes: How do data teams decide which transformations to push down to dbt and which to leave up in the notebook?
In this panel discussion, analytics engineers, data engineers, and data scientists discuss what transformation means to them, where and when transformation happens in their stack, and how to collaborate effectively between high- and low-level forms of transformation. The goal will be to surface the mental models of data transformation, from each perspective, in order to help data teams draw their own lines. There is no one-size-fits-all definition of transformation, and this discussion explores many branches of the topic.
Data Scientist and avid coffee drinker