Speaker

Brian Olsen

Brian Olsen

Head of Developer Relations @ Tabular

Chicago, Illinois, United States

Brian is a U.S. Marine turned software engineer and developer advocate working to foster the open-source Iceberg and Trino communities. Brian spent four years as a data engineer at a cybersecurity company working on pipeline maintenance and query optimization. While in this role, Brian was responsible for maintaining data pipelines and migrations to include replacing some legacy data warehousing systems to use open-source Trino. Brian is a published author in ACM and IEEE geospatial database conferences

Area of Expertise

  • Information & Communications Technology

Topics

  • Big Data Analytics
  • Trino
  • Databases
  • data engineering
  • Distributed Databases
  • Cloud analytics
  • Software Development
  • Open Source Software
  • Software Engineering

Hydrating real-time data in Pinot with fast batch queries on Trino

In the spectrum of real-time to batch systems, there has never been a perfect solution that can be found that addresses the limitations of both extremes well. Some systems that fall between, typically only do one or the other well and can't be adopted for both. Pinot is state-of-the-art for real-time ingest and analytics streaming, but has common limitations any real-time systems have. Trino is well known for being an incredibly fast batch analytics query engine that enables not only large-scale batch processing but also runs federated joins across multiple tables across multiple data sources.

This talk covers the many benefits of running Trino and Pinot together like using Trino as a unified point of access for reads and executing writes into Pinot and interactive analytics over data in Pinot and cloud storage. This talk will also dive into recent updates in Trino's Pinot connector like optimizing the efficiency of reads; putting less load on Pinot brokers. Learn how to augment your real-time architecture with this ludicrously fast batch query engine.

Trino: A ludicrously fast query engine

You may be familiar with the Presto plugin used to run fast interactive queries over Pulsar using ANSI SQL and can be joined with other data sources. This plugin will soon get a rename to align with the rename of the PrestoSQL project to Trino. What is the purpose of this rename and what does it mean for those using the Presto plugin? We cover the history of the community shift from PrestoDB to PrestoSQL, as well as, the future plans for the Pulsar community to donate this plugin to the Trino project. One of the connector maintainers will then demo the connector and show what is possible when using Trino and Pulsar!

Brian Olsen

Head of Developer Relations @ Tabular

Chicago, Illinois, United States