The era of siloed data and slow batch processing is over. This session will guide you through building a powerful, real-time data platform that empowers agentic AI applications to reason over both structured and unstructured data.

I will tackle the real-world challenge of unifying diverse data sources—from streaming Kafka topics and REST end points to large, unstructured datasets—and transforming them into a single, searchable knowledge base. Discover a modern architecture centered around Snowflake, leveraging cutting-edge tools to ingest and process data with speed and precision.

I will provide a comprehensive, end-to-end blueprint, demonstrating how Snowflake OpenFlow (Apache NiFi) augmented with some Python acts as the glue for this architecture. We will also showcase how Snowpipe Streaming v2 can be used to efficiently load data from a real-time pipeline. A concrete example of this architecture will be provided, referencing a project that demonstrates a real-time AI application.

Github
For a concrete example of this architecture
https://github.com/tspannhw/TrafficAI/blob/main/README.md