Session
The SQLite Moment for Analytics: Building Lightweight Data Pipelines with DuckDB
Point of the talk: To show developers how to eliminate costly, complex cloud data warehouses for medium-scale datasets by embedding DuckDB directly into Python applications for serverless, zero-copy data processing.
Duration: 45 Minutes
Detailed Breakdown: Many data pipelines unnecessarily spin up heavy, distributed clusters (like Spark) to process 10–50 GB datasets, incurring high financial costs and latency. This talk introduces DuckDB as an in-process, columnar analytical engine that lives inside your Python runtime. We will explore how to write ultra-fast SQL directly over local or remote Parquet, CSV, and JSON files. Attendees will learn the architecture of zero-copy memory sharing between DuckDB and PyArrow/Polars, ensuring seamless data handoffs without serialization overhead. Finally, we will build a local-first dashboard pipeline that runs entirely on a single machine while matching the speed of a cloud warehouse.
Muhammed Mizaj
Product Engineer at UST Global
Thiruvananthapuram, India
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top