Session

Streamifying Reference Data for Temporal Consistency with Telemetry Events

Reference data provides critical context for interpreting telemetry, yet its traditional static handling limits temporal consistency in dynamic systems. Unlike real-time telemetry, reference data is often updated asynchronously in bulk, making it difficult to align with rapidly changing events. For example, transit systems rely on GTFS files for schedules and GTFS-RT for real-time updates, creating mismatched formats and update frequencies. To address this, reference data must be "streamified," where every state change is treated as an event. By detecting and signaling changes—such as through hashing techniques—reference data can become temporally consistent and integrate seamlessly with telemetry for point-in-time accuracy.

Reframing reference data as event streams enables a unified approach to real-time processing. Standardizing reference data events with CNCF CloudEvents embeds critical metadata for temporal alignment, ensuring consistent context for both real-time and historical analysis. This transformation unlocks new potential for industries like IoT, transit, and healthcare, where accurate, time-aligned reference data is essential. By treating reference data as a first-class citizen in event-driven systems, organizations can achieve higher precision and reliability in their analytics.

Clemens Vasters

Principal Architect, Messaging and Real-Time Intelligence, Microsoft

Viersen, Germany

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top