Accelerate your Mapping Data Flow with Flowlets

We are often faced with the challenge of having to implement the same code over and over again. It can be time consuming to repeat the same operations again and again. We can sometimes find ourselves struggling to remember the exact pattern of implementation or forget to check that important checkbox that makes everything tick just as it should. In Synapse Analytics mapping data flows we can implement small prebuilt implementations of transformations which offer repeatable high quality code every time.

The benefits of using Mapping Data Flow Flowlet are numerous. Aside from offering real time savings they also provide consistency so that each implementation is the same. This improves code reviews and build quality. It allows junior developers to implement patterns built by senior and principal developers with the knowledge that everything in the flowlet is correct and robust leaving time to complete checks on custom elements. Further more the nature of the mapping dataflow data drift technologies allows for data agnosticism. Prior to the flowlet and after the flowlet data is known but the processing in the flowlet does not rely on any schema being present within the code.

Attendees of this session would benefit from a knowledge of Synapse Analytics and mapping data flows as well as an understanding of the parquet data format. Knowledge of Data Warehouse terminology and understanding of type 2 slowly changing dimensions would be advantageous.

The session will describe mapping dataflows, the patterns for transforming data and how to build mapping dataflows for processing data as a SCD2. It will explain how to implement these flows into a mapping data flow as repeatable transformations and how to import mapping data flow flowlets into other synapse workspaces.