Side Effects Are Why We Can’t Have Nice Things

What will save us from software complexity? Microservices architectures? Immutable data? Functional languages? Event streaming? These may seem like disparate solutions, but they’re all connected, They’re all striving to solve the same fundamental problem: Side effects.

Side effects are one of the most insidious causes of software complexity, and yet we’re barely aware of them. And the attempts to tackle side effects have produced some of the most interesting developments in software development in the past decade, yet as an industry we’re barely aware of the fundamental pattern tying everything together. Let’s fix that.

Join me to understand what side effects are and why they matter. You’ll learn to spot them in your own code. You’ll see how they sneak into our tests, our APIs and our systems’ designs and make everything harder. Then we’ll look at our industry’s many solutions to side effects. You’ll see why immutable data structures are appearing in every language; why we invented OO and why it might get replaced with FP. And we’ll see what it all means for the larger designs of microservices, data processing and event systems.

Side effects are a hologram - put the right light on them and you can see a whole picture of modern computing emerging. Take a look and you’ll see the nice things the future could hold.

Slaying a UI Antipattern

Fetching data from the server should be one of the easiest things frontend programmers do. And yet even big names like Twitter and Slack make a simple mistake that consistently spoils the UX of their sites.

In this talk we’ll look afresh at the simple act of loading remote data, and how it’s just slightly more complicated than people think. Then we’ll see how Elm’s excellent data modelling capabilities let you capture this complexity and banish the bug permanently.

You’ll learn how to improve the user experience of every HTTP call you make, and pick up some wider ideas on how Elm’s type system can guide you to clearer, more useful designs that make for better software.

A Beginner's Guide to Realtime Data With Kafka

Apache Kafka's an interesting beast. It solves many different data-processing problems by being part queue, part database, and part message bus. But it can be daunting to get started, and like many things, the only real way to understand Kafka is to build something with it. So let's do that!

In this live-coding session we'll take our first steps with Kafka, building a system that tracks the live status of metro trains. As we do so, we'll build up an understanding of some of Kafka's core ideas - topics, connectors, consumers and groups - to build up a mental model of how Kafka works, and where it works best.

(And for those of you who already know Kafka, you'll learn some modern JavaScript and see how quickly you can build a useful app.)

By the end you'll have a starter project, and the shape of Kafka in your mind so you can make faster progress building your own realtime applications.

Types As Design Tools

Types are much-discussed as a tool for correctness. Are they vital for a robust system, or has TDD made them obsolete? Do types help you write correct programs, or do they just restrict expressiveness?

What's not talked about nearly enough is how types are a window into your system's design. A good type system can reveal architectural secrets when you've barely even looked at the code. And it can reveal design flaws and opportunities for improvement at a glance - if you know how to look.

With a collection of examples gathered from real live systems, we'll look at how the type signatures of existing programs are trying to guide you to better designs.

PICKUP DATA - A Kafka Adventure Game

Apache Kafka is used by many video gaming companies to manage and analyse live data from their players, handling billions of events across dozens of countries every day. But none of them have thrown caution to the wind and used Kafka as their core game engine.

In this talk we'll revisit the quintessential video game, the Text-Based Adventure, and implement as much of it as we can in pure Kafka. We may not break the Steam sales records, but along the way we'll learn a lot about the building blocks of event systems, some interesting Kafka Streams tricks, and get a glimpse of how many different needs a good event store can satisfy: data collection, processing & enrichment, analytics, real-time reporting and more.

Livecoding a Kafka Dashboard

How much of a genuine realtime system can you build in a lunchtime's worth of programming? In this live-coding session we'll build a real-time, Kafka-driven sales dashboard from scratch and see how you:

- Set up a Kafka Consumer with Python.
- Easily handle real-time Stream Processing with ksqlDB.
- Build out a WebSocket server to broadcast live data to your users.
- Create a basic React app to visualise data in a way anyone can access.

We'll start with an empty directory and by the end, you'll have all the foundational pieces of a dashboard that could serve KPIs to everyone in your organisation, or just form the basis of your next lunchtime hacking session.

Communicating in Types

Modern type systems have come a long way from the days of C and Java. Far from being nit-pickers that berate us for making mistakes, type systems like the ones found in Haskell, PureScript and Elm form a language in themselves. A language for expressing high-level ideas about our software to our colleagues and to the computer. A design language.

In this talk, we'll take a look at the way the right kind of type signatures let us talk about software. We'll survey how to state our assumptions about the domain we're coding in and how each part fits together. We'll show how it can highlight problems, and surface opportunities for reuse. And most importantly, we'll see how types can help you communicate to your coworkers, and to future maintainers, with little effort.

You may have heard the phrase, "programs are meant to be read by humans and only incidentally for computers to execute." Come and see how a modern type system is about communicating ideas to humans, and only incidentally about proving correctness.

Kafka Performance Tuning, But Backwards

I have a simple streaming application that comfortably processes a million records a second against a modest Apache Kafka® cluster running on the far side of the Atlantic ocean. This is clearly too powerful, too efficient, and I risk making myself redundant. Where’s the job security in a high-performing system?

In this code walkthrough we’ll look at the most important performance-tuning settings for Kafka and see how we can easily destroy throughput, add latency, and bring the system grinding to a halt by setting the right parameters to the wrong values.

By the end of the session you’ll know exactly which configuration settings you should look at if you want easy ways to ruin the performance of your own streaming applications. Alternatively, you could use these tips to speed things up, if you’re into that…

From Metrics To Music: Making Connections with Apache Kafka

Apache Kafka® is great at joining different software systems and processing their data in real time. But while event-based approaches like Kafka are growing in popularity, it can still be hard to see how a whole system fits together. Let’s fix that by exploring a complete Kafka application. One that combines two domains that no one ever expected to be integrated: Electronic music and the London Underground.

We’ll explore the use of Kafka Connect to make sourcing London’s transport data trivial. We’ll use Kafka Streams to live-manipulate that data into a form we want to present. And to get away from the usual charts and maps web tier, we’ll send our data out to a more exotic data source - a group of synthesizers that represent a well-running system by playing in harmony (and represent a broken one with something more discordant!)

You’ll see—and hear—how the whole thing fits together and get a better sense of what a complete Kafka solution looks like. You could use the same recipe as the backbone for a more traditional business dashboard, or maybe the next time there’s a crisis at work, you could have it accompanied by its own live backing track…

This session includes music, so I'll need some extra help from the PA team.