Session
High-Performance Full-Text Search on Apache Iceberg Using Lucene-Integrated Indexing
Modern data lakehouses built on Apache Iceberg deliver massive scalability and cost efficiency, but they are fundamentally optimized for scan-heavy analytics—not for sub-second, full-text search over terabytes of cold data. This creates a growing gap for teams who want search-engine-like experiences directly on their data lake without duplicating data into a separate system.
In this session, we explore how Apache Pinot bridges that gap by integrating Apache Lucene indexing directly into its distributed serving layer, while keeping Iceberg (Parquet on object storage) as the system of record. By virtualizing Pinot segments over Iceberg files and selectively pinning Lucene indexes to local NVMe storage, Pinot enables high-performance inverted-index search without large-scale data movement.
Attendees will gain a deep understanding of the architecture, indexing strategy, and I/O optimizations that make interactive full-text search possible on multi-terabyte Iceberg tables.
Xiang Fu
CoFounder & Chief Architect @ StarTree
San Francisco, California, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top