Session
Building a Data Lakehouse with Dremio, DBT, and Apache Iceberg
Organizations are increasingly moving towards data lakehouse architectures to combine the flexibility and scalability of data lakes with the management features of traditional data warehouses. This talk introduces a cutting-edge approach to building a data lakehouse utilizing Dremio, DBT (Data Build Tool), and Apache Iceberg, offering attendees a comprehensive blueprint for implementing a scalable, efficient, and cost-effective data platform.
We'll start by exploring the fundamentals of the data lakehouse architecture and the unique benefits it provides over conventional data storage solutions. The focus will then shift to how Dremio acts as a core engine, enabling lightning-fast query performance directly on data lake storage, thus eliminating the need for costly and complex data movement and duplication.
The integration of DBT with Dremio will be a major highlight, detailing the "why" and "how" behind leveraging DBT for data transformation within the Dremio environment. We'll discuss how this combination facilitates a more agile and collaborative workflow among data teams, streamlines the development process, and ensures data quality and reliability across the enterprise.
Apache Iceberg's role in this architecture will also be dissected, illustrating how its open table format plays a pivotal role in managing large-scale analytic datasets with high concurrency and providing schema evolution without performance penalties.
Attendees will leave with actionable insights on:
- Designing a scalable data lakehouse architecture that aligns with business goals.
- Leveraging Dremio for optimized query performance and cost savings.
- Implementing DBT in a Dremio environment to enhance data transformation processes.
- Utilizing Apache Iceberg to manage data at scale and ensure consistency and reliability.
Alex Merced
Senior Tech Evangelist for Dremio
Atlanta, Georgia, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top