Ben Gamble
Technology Sommelier, AI Whisperer
Cambridge, United Kingdom
Actions
Ben Gamble is Field CTO at Ververica. With a background in engineering leadership and entrepreneurship, he’s led teams across logistics, gaming, and mobile apps, focusing on solutions involving GPS, AR, and multi-user collaboration. Off-duty, he enjoys creating video games and trading cards, and cares for his two children and three pet ducks.
Links
Area of Expertise
Topics
A match made in Data: matching analytics data with realtime usecases
Matchmaking is matching demand, with the right supply to facilitate a good transaction, ei a person wanting a taxi with one going in that direction, or the right team in a game.
With some worked examples, we’ll walk through ingesting data, capturing user intent, and building real time analytics pipelines with Apache Kafka & Apache Flink and Clickhouse. Then we'll see how we can blend that with transactional data to drive user actions while handling real world problems such as flaky users, sudden traffic jams, or losing WiFi, .
One of few universal truths is that people prefer not to wait. Whether they're looking for a game to play, the next episode to watch, a taxi home, or even a date, there is a finite amount of time you can delay a user for before they will leave your service. However if the match is not good enough, that can actually be worse.
You don't want to match someone in London with a taxi in Tokyo, or a new player against a world champion. Also, history can be important - trying to book a taxi to travel outside its normal working area will not be popular.
Good matches rely on many factors, some being hard constraints, and others which are weighted scores. Some of these can vary in real time. You don’t match someone in London with a taxi driver in Tokyo or a new player against the world champion. In addition, you’ll want to know what a user has done in the past so you can inform that match with past behaviors like balancing a team for a game, or matching people based on likely first date locations.
So lets explore streaming joins, Batch pre-aggregation, and dealing with rage quits in an open source architecture designed to scale out across the globe
VERA: The Engine Revolutionizing Apache Flink
Apache Flink is one of the most popular stream processing frameworks in the world, used by nearly 2k companies for real-time analytics and applications. But since the publication of State Management in 2017, workloads, deployments, and storage models have changed, evolving significantly, while the State Management architecture has stayed relatively the same.
Enter VERA: the cloud-native engine built to adapt OS Flink into the modern era, specifically to address those evolving changes and advancements in workloads, deployments, and storage models.
In this talk, we’ll explore what VERA is, how VERA helps to democratize stream processing, how VERA addresses OS Flink limitations and how VERA ultimately helps businesses to solve both batch and streaming use cases.
We’ll explore the three core pieces (pillars) of VERA: Streaming Data Movement, Real-time Stream Processing, and Streaming on a Lakehouse, and how they are tailored for seamless support of data ingestion (connectors), stream processing in real time (Flink), and streaming Lakehouse architecture.
Specifically, we’ll discuss how VERA tackles the challenges of:
*Evolving Apache Flink to run natively in modern cloud environments
*Providing updated fault tolerance; i.e Flink’s checkpointing
*And how VERA solves inherent challenges with large stateful applications
By the end of this session, you’ll understand what VERA is, why it was built, and how VERA revolutionizes Apache Flink.
My tokens are limited. You must use the right context: An agent builder's guide
AI models can only process so much information at once. This hard constraint—the context window—limits their ability to make smart, context-aware decisions. The solution isn't waiting for bigger models. It's becoming deliberate about what you feed them.
This talk explores how to give AI agents precise, useful context by treating token budgets like any other scarce resource. As a wise person might say, "An AI only knows what it's shown."
Priority-Based Context Management: Not all context is equal. MoSCoW prioritization (Must/Should/Could/Won't) ensures critical information survives while noise gets discarded. Temporal relevancy scoring means recent context outweighs stale data.
Smarter Retrieval: Moving beyond basic RAG—exploring open source vector databases, hybrid search strategies, and knowing when similarity alone isn't enough.
Stateful Agents: Building systems with frameworks like LangChain that adjust context on the fly, remember what matters across sessions, and adapt to preferences over time. Two-tier architectures using tools like Redis and PostgreSQL let agents recall history without burning tokens.
You'll leave with practical patterns for building AI agents that work with sharp focus—using only the data that matters.
Escape the Micro-Maze: Build Fast, Scalable Streaming Services with Apache Flink
So, you're building microservices, and if you're like me, you've probably found yourself wrestling with Kubernetes, trying to manage state, handle failures, and figure out scaling for each service. Someone inevitably says, "Just build it stateless!" and I always think, "I'd love to see that work seamlessly in the real world." I believe there's a more straightforward way to build fast, resilient user experiences.
In this talk, I want to share a somewhat radical idea for those of us tired of the traditional microservice shuffle: building our operational logic, and even entire microservices, directly in Apache Flink. I'm not just talking about data pipelines; I'm proposing we start "going operational with Flink," moving beyond its traditional analytical domain.
I'll dig into why I think Flink offers a distinct advantage for application development. First, Flink was born for state, and I'll show you how its robust state backends can simplify what's often a major headache in microservice architectures. Then, we'll look at how Flink's inherent fault tolerance and scaling mechanisms can apply to our application logic, not just data processing – meaning less ops and more dev for us. Finally, I'll discuss practical approaches for handling point-to-point calls, comprehensive state management, and general application development patterns within Flink. I've come to think of Flink as an application server, supercharged for streams and state.
Join me to see how Apache Flink can simplify our architectures, make our user experiences faster, and potentially let us bid farewell to some of those microservice complexities. And with a bit of help From Kafka streams, we'll see it action
Avro, arrow, protobuf, parquet and why
When it comes to streaming data, there's a lot of data formats out there, and as much as we give JSON a pass because it's easy, it's an overhead we can't often afford, so let's dive in and see which ones matter and why!
Come along and we'll run through the how and why of data serialisations, how to manage schemas and why we need rows and columns in streaming, and in data lakes.
We'll talk through tools evolution, efficiency and portability. Some horror stories of data conflicts and some of the newer ways to take advantage of these
A Records ACID trip through kafka
A Record’s ACID trip through Kafka
In this talk I will be exploring how to use Apache Kafka’s in-built transaction API to build transactional microservices, allowing distributed systems to meet processing guarantees.
If there's one thing we know about software, it is that eventually something will fail, and you many lose data.
This is not the end. Designing for failure has brought us many useful innovations like the TCP protocol, the Erlang programming language, and even Apache Kafka itself.
It's so important, that databases have enshrined these properties as ACID compliance.
But what happens when there is more than one system in your transaction? Classically microservices have to do more than just commit changes to one database or one Kafka topic. In addition, how do you maintain exactly once processing guarantees when you may have to deal with systems failing?
Sometimes it's more than just a two phase commit, and when you are dealing with payment systems, being able to act upon a stream continuously while maintaining ACID characteristics and exactly once semantics is mandatory.
Follow along as we see what happens when systems start to degenerate, and what we can and can’t trust at scale. Learn about when to use Kafka as the transaction controller, what can and can’t be stateless, and what are the tradeoffs.
We will explore completion criteria, the routing slip pattern, the outbox pattern, and others as we go on a trip through the various methods of ensuring ACID (atomicity, consistency, isolation, and durability) compliance and exactly once processing, in an asynchronous distributed system. Leave with a few extra tools and patterns about how to make large scale systems reliable
Going Multiplayer with Kafka
Today we’ll walk through building multi-user and multiplayer spaces for games, collaboration, and for creation, leveraging Apache Kafka® for state management, and stream processing to handle conflicts and atomic edits.
We’re not building the metaverse! But as technology matures, ideas jump between disciplines, from ReactJS using ideas from game rendering, to the recent innovations in ECS patterns that borrow heavily from column stores in databases - there’s never been a better time to bring ideas from one sector of software engineering to another. Apache Kafka® makes event management simple so what can we borrow to make it collaborative?
Starting with a simple chat application and working into cursor sharing, collaborative editing and even a multiplayer game, we’ll walk through how to collect and manage user inputs,, and how backing onto an event log allows for version control, undos and time travel. We’ll also explore the various ways you can build up a canonical source of truth in a distributed system, from snapshots, to lockstep sync, to eventual consistency. Along the way we’ll learn a bit about CRDTs (conflict-free replicated data types), mergeable data structures and some of the ways to manage this complexity effectively.
Going AsyncAPI: the good, the bad, and the awesome
In this talk, I’ll explore the good, bad, and awesome aspects of building Async API into our open data hub. As advocates of open source tools, it is our mission to simplify the collection and distribution of streaming data by taking care of everything under the hood, including business-to-business exchange of data and “last mile” delivery to end consumers.
Beginning with a discussion on open API, I’ll walk you through our deliberations, and why we chose AsyncAPI, how it helped us and what it cost. I’ll tell you how we improved our tools to make use of AsyncAPI specs; how we managed the gaps in the specification; along with the benefits.
AsyncAPI spun out of open API with a goal to solve some of its shortcomings. The initiative set out to standardize asynchronous and event driven APIs across the industry. With the proliferation of IoT devices and the connectivity promised by 5G, having standard ways to connect has become more important than ever.
AsyncAPI has been added to every product we host on our open platform. Why? Because we believe AsyncAPI is a good standard for open event based data/APIs, and we want to support a proper way to carry out code generation and validation - with specifications that make sense.
Thousands of software engineers around the world have provided code, documentation, tests, or other improvements to open source projects. With the help of initiatives like AsyncAPI we want to help people liberate their data by tackling the common challenges they face when trying to distribute it.
Current New Orleans 2025 Sessionize Event
Flink Forward 2025 Barcelona Sessionize Event
Data Streaming Summit Virtual 2025 Sessionize Event
Flink Forward Berlin 2024 Sessionize Event
Current 2022: The Next Generation of Kafka Summit Sessionize Event
Ben Gamble
Technology Sommelier, AI Whisperer
Cambridge, United Kingdom
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top