Session
Real-Time Data Platforms at Trillion-Event Scale
Some of the largest real-time platforms in production today operate at scales that were considered theoretical a decade ago. Numbers like Trillion events are not unique — Security, AI agent platforms, observability backends, and large SaaS analytics systems are all pushing into the same territory. The interesting question is what actually breaks at that scale, and which architectural choices keep the system honest.
This talk distills patterns and failure modes from years of building streaming and AI data platform infrastructure, framed against publicly disclosed industry references rather than any single employer's internals. Topics include:
- Stateful streaming joins across different data streams
- Handling replay storms and out-of-order events without latency collapse
- The honest tradeoffs between latency, cost, and correctness — and when each one wins
- Observability and degradation modes that keep the platform usable when something is failing
Attendees will leave with concrete guidance on designing real-time systems that fail loudly, recover predictably, and surface the metrics that actually matter at scale.
Note: Views are my own and do not represent any employer. Examples reference publicly disclosed sources.
Diptamay Sanyal
Principal Engineer | Data, AI & Cybersecurity Platforms | IEEE Senior | Forbes Technology Council | ex-CrowdStrike, Hubspot, Dataminr, Axon
Bayonne, New Jersey, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top