How and where to define metadata for a metadata-driven framework often feels like choosing between lesser evils: verbose YAML files that punish bad indentation, sprawling JSON blobs, or fragile SQL tables only touched by brave souls. Some teams go so far as to build custom apps to manage metadata, which often feel like relics from 2005.

But what if the solution is something already in everyone’s toolbox? Enter Excel—an often-disparaged tool that’s both beloved and cursed, yet undeniably familiar and accessible regardless of its database-status. In this session, we’ll explore how Excel can serve as a structured, accessible entry point for defining the required metadata for an end-to-end ELT pipeline following the medallion architecture—without needing to build a full app or ask business users to learn Git.

We'll demo how OneLake File Explorer and open mirroring allow seamless syncing structured Excel inputs, including robust metadata validation, without the complexity of custom apps. While Excel is a great starting point, certain metadata—like dynamic pipeline variables and execution and auditing logs—are better handled by Fabric Databases. We’ll illustrate how this hybrid approach keeps things scalable and robust.

Of course, making Excel work for this isn’t just “save-as CSV and hope for the best.” We’ll demonstrate how to avoid turning Excel into yet another workaround. Moreover, we’ll tackle real-world challenges, such as how to ensure metadata entries follow the required schema, validate them effectively, and manage metadata versioning, to ensure that your pipeline remains robust and fully traceable.

By the end of this session, you’ll be enriched with an alternative—albeit slightly controversial—metadata-driven framework that balances flexibility with structure, showcasing that Excel doesn’t have to be the villain in your data story. By integrating Excel, we enable domain experts and business users to take ownership of their metadata, enhancing collaboration without sacrificing data quality or governance.