Session
A Holistic Model of Data Contracts for Better Data Governance
Data contracts between data products are often viewed one-dimensionally as promises made by data providers to their consumers, e.g. by stating SLOs. However, in actual large-scale data landscapes the relations between data providers and consumers are more complex: First, consumers should also state their expectations towards the producers (which is already common practice with consumer-driven contracts in microservice architectures). Second, providers often have expectations as well, e.g. that consumers will adhere to relevant policies when sharing sensitive data with them. Third, consumers should be able to state promises about how they will handle the received data.
Unfortunately these other three dimensions of data contracts are often either not documented explicitly or are overlooked entirely, which increases the risk of data incidents. Also, because these dimensions have mostly been overlooked so far, current data mesh platforms lack the support to define, manage, and verify such complex data contracts. We present a new model of data contracts that incorporates the missing dimensions and thereby allows for data mesh platforms to support data governance in a more holistic way. To demonstrate this, we have implemented support for such complex data contracts in a popular data mesh platform and assess the potential impact of our new model with an industry case study.
Dr. Simon Harrer
Co-Founder and CEO, Entropy Data
Gerolzhofen, Germany
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top