Gonzalo Ortiz Jaureguizar
Performance engineer at Startree
Madrid, Spain
Actions
I am a software engineer specialized in developing databases in Java. I love understanding how libraries and frameworks work under the hood and to design and implement high-performance systems. I have worked on prototypes such as ToroDB, the first Spanish database unicorn Devo and since 2022 I'm working at StarTree as an Apache Pinot contributor.
Area of Expertise
Manage memory in the JVM as if it were C
As in many other languages, in Java heap memory is managed. That is, the program explicitly reserves memory, but does not indicate when to free it and delegates it to the garbage collector.
This way to deal with memory has several advantages, but it also has some drawbacks that become more problematic when the program has to work with a large amount of data or process it very quickly. For example systems like Apache Kafka or databases like Apache Pinot largely avoid using managed memory and instead manually allocate and free memory in what is known as offheap memory.
This is a technical and practical talk about how to use this memory in the JVM, when it's worth using it, and how it affects our code and deployments including examples in real applications and libraries.
Deep Dive into Apache Pinot’s Multi-Stage Query Engine: Architecture and Performance
Apache Pinot is an open-source, real-time distributed database for low-latency, high-throughput queries on large-scale datasets. As industries like e-commerce and IoT demand real-time analytics, Pinot's original single-stage query engine (SSQE) struggled with complex queries involving joins and window functions.
The Multi-Stage Query Engine (MSQE) addresses these challenges, enabling advanced relational operators and supporting complex query execution. In this session, we’ll delve into MSQE’s key innovations, including its integration with Apache Calcite for smart query planning and gRPC for efficient inter-server communication. We’ll also explore strategies for optimizing data shuffling, thread management, and query execution statistics to scale across large environments.
Join us to discover how MSQE extends Pinot’s capabilities, overcomes its limitations, and transforms it into a robust solution for modern data analysis.
Gonzalo Ortiz Jaureguizar
Performance engineer at Startree
Madrid, Spain
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top