Over the past decade most companies transitioned to microservice based architectures, successfully applying domain driven design techniques to their applications and services. But, while some engineers were decomposing monoliths into microservices, others worked hard to create the biggest monolith of them all: the data lake. Although this centralized data platform model works for use-cases with simpler domains, it falls short whenever we have to deal with a large number of data sources, multiple teams, diverse consumers, and changing data requirements. The latest iteration in distributed data architecture is known as a Data Mesh and it solves many of the ills of previous centralized approaches.

In this talk we’ll transfer the lessons we learned from microservices to data architecture. How to apply domain driven design and product-oriented thinking to data management. How to make our data products self-describing, discoverable, and secure. How to build tests to ensure data quality and completeness. And what technologies and techniques can help us achieve our vision of agile, distributed data architecture.