Will LaForest
Global Field CTO, Confluent
Vienna, Virginia, United States
Actions
As Global Field CTO for Confluent, Mr. LaForest collaborates with customers worldwide across all industries, guiding them in leveraging the advantages of a data streaming and event-driven architecture. He is passionate about data technology innovation and possesses over 30 years of experience assisting clients in managing data. His technical expertise spans many diverse areas such as software engineering, data management and science, machine learning, and statistical visualization. Mr. LaForest holds degrees in mathematics and physics from the University of Virginia.
Links
Area of Expertise
Topics
Getting Gory with Active-Active Multi-Region Disaster Recovery
Kafka is one of the most durable and reliable data systems in existence. Thousands of mission critical businesses have been ironing out the wrinkles over the last decade. Once you move your event driven apps or streaming pipelines out of the confines of a single region the route is not that well understood. Asynchronous replication and eventual consistency add several challenges irrespective of the technology you are using. Businesses have a strong desire to employ an active-active strategy to gain the benefits of data locality and to mitigate the blast radius of regional failures. When you sum this all up, you have yourself a complex challenge. In this talk, we will cover:
* The multi-region replication options and their pros and cons
*Differences in streaming applications that drive design decisions
* Pitfalls in application fail-over and fail-back
* How to handle event oriented considerations like event ordering, exactly once, and aggregate state
Although there are robust technologies in the Kafka ecosystem for multi-region replication, applications must be designed appropriately to achieve the benefits of active-active deployments. At the conclusion of this talk, you will be armed with patterns to help you achieve business continuity and the benefits of regional data locality with your data streaming applications.
Building a Dynamic Rules Engine with Kafka Streams
The benefit of real-time data can be measured by how frequently the data in question changes, nowhere is this more apparent than threat detection. Responding to an ever changing landscape of attacks and exploits requires a system that can not only handle the scale and dynamic nature of the data but also a dynamically changing set of detection rules. We developed Confluent SIGMA, an open source project built on Kafka Streams for the open SIGMA DSL, to handle real-time rule additions and modifications. In this talk we will cover:
* The architecture of our Kafka Streams layer that makes it possible to use external data feeds as rule input
* How we handle dynamic criteria for joins and filters
* Best practices for writing dynamic rule engines in Kafka Streams
* Upcoming improvements to Kafka Streams to support versioned rules
Although Confluent SIGMA focuses on cyber threat detection this same pattern can also be applied to any DSL (domain specific language) that would benefit from real-time stream processing. After attending you will have the framework to drive dynamic rules through Kafka Streams for any use case that might require it.
Where in the world is Franz Kafka?
Apache Kafka is the de-facto standard for event streaming and creating data pipelines that can feed a variety of different tools. It is very common for the data to have geospatial characteristics but to date there has been relatively little work done around how to leverage this natively in Kafka. The common approach is to just dump all the data into some geospatial store or toolset and do retrospective analysis and queries. This of course loses all the advantages of handling it in realtime before it ever goes to an external tool. In this talk I will discuss the creation and demonstrate the usage of geospatial UDFs in ksqlDB. I will also talk through the advantages of doing geospatial processing directly in Apache Kafka.
Will LaForest
Global Field CTO, Confluent
Vienna, Virginia, United States
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top