This lightning talk will do a quick dive into the metadata layer of Iceberg to cover these topics:
- Overview of Fabric/Iceberg internal metadata tables (essentials for time travel)
- Time Travel queries like
Select * from db.table.history;
Select * from db.table.snapshots;
Select * from db.table.files;
Select * from db.table.manifests;
Select * from db.table.partitions;
- Advanced Topics:
a. Rollback
b. Maintenance - e.g., compaction (rewrite_data_files), remove orphan files, expire snapshots
- CoW (Copy on Write) vs MoR (Merge on Read)
a. Default - V2 Copy on Write
b. V2 Merge on Read
c. V3 Merge on Read (the best but query engines like Dremio does not seem to support this yet)
- Nessie branching: branching at catalog

At the end of the talk, participants will leave with a better understanding of Fabric/Iceberg time travel and maintenance features.