This session, led by Marco Gorelli and Alessandro Romano, offers a comprehensive overview of the evolving landscape of dataframe libraries in Python, focusing on pandas, Polars, and emerging solutions for dataframe standardization. We'll begin with an introduction to pandas, highlighting its core features and well-established place in the data science ecosystem. While pandas has enabled rapid data manipulation and analysis, we will examine their performance limitations and the trade-offs associated with large-scale data processing.
The talk will then introduce Polars, a rising alternative to pandas. We will explore its unique architecture, optimized for high-speed performance, and benchmark its capabilities in comparison to pandas across common workflows. Following this, we’ll discuss the challenges of dataframe standardization, examining how varying implementations across libraries impact code interoperability and usability in diverse data environments.
Finally, we’ll present Narwhals, a Python library designed to bridge dataframe differences by providing a unified, agnostic approach to dataframe operations. This library simplifies multi-library workflows, enabling users to work seamlessly across different dataframe implementations. Join us for a deep dive into the present and future of dataframe manipulation, with practical insights for boosting efficiency and agility in data science projects.