Session

Insecure By Design: How Not to Build Your Next Data Pipeline

Best practices abound for data pipeline design. Summarizing optimal strategies is helpful, but learning from negative examples provides a unique perspective for improving processing practices. Derived from real-world experiences, this presentation spotlights several pitfalls from production data pipelines, with the goal of helping you avoid the same mistakes. Streaming systems built around idealized inputs and consistent data rates may perform well initially, but what happens when an outsized file hits the pipeline? Structured records enable scalable processing, but what if the format itself allows dangerous operations? Parameterization is a core feature of software interface design, but maximum flexibility in data pipeline design leaves the door open for misconfigured or malicious inputs. Considering known design flaws today empowers practitioners to build more secure pipelines tomorrow. Focusing on Apache NiFi, this presentation covers concepts that apply to many streaming platforms.

David Handermann

Apache NiFi PMC Chair, Senior Software Engineer at Snowflake

San Antonio, Texas, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top