Andrii Raikov
Principal Software Engineer at Delivery Hero SE
Berlin, Germany
Actions
Andrii has a total of 17 years of software engineering experience and has been very passionate about Go for the past 6 years. Over the past 5 years at Delivery Hero, he has adeptly harnessed Golang, excelling in constructing high-load applications, leveraging Golang's concurrency capabilities to engineer scalable solutions that optimise performance and handle substantial data processing tasks.
Links
Area of Expertise
Topics
Time to the Backend: The Hidden Delay in Your Application's Request Journey
This talk was inspired by Jonathan Amsterdam’s presentation on HTTP Routing Enhancements at GopherCon EU 2024. He discussed the latest improvements in HTTP routing and the ongoing community pressure to maintain performance. One remark he made really stuck with me: "Why are you so concerned about milliseconds of HTTP routing when you’re dealing with JSON?" It struck a chord because it made me realize that the real battle for milliseconds has already been lost elsewhere. That’s when I knew I needed to present our "Time to Backend" metric—so others can clearly understand that focusing on micro-optimizations in routing doesn't address the bigger latency issues that impact user experience.
Have you ever wondered what happens to your request when you hit a button in your application? In this talk, we’ll dive deep into the often-overlooked aspect of application performance—the "Time to Backend." Drawing from real-world incidents where application response times were unusually slow despite seemingly healthy system metrics, we’ll explore how network and mobile provider delays can overshadow code optimization efforts.
You’ll learn how, even with perfectly optimized back-end systems, user requests can take several seconds to traverse networks and reach your service. Through the introduction of the "Time to Backend" metric, we’ll demonstrate why it's crucial to focus on larger network issues that impact user experience more significantly than micro-optimizations in your code.
Join us as we discuss the key takeaways from dealing with these complex incidents, how this metric led to the creation of better monitoring systems, and the lessons learned from not just looking at your own infrastructure but considering the broader mobile ecosystem.
AI-Driven Load Test Analysis: From Routine Toward Action
Scalability and reliability are key factors, especially when handling millions of daily orders. Load testing is essential for ensuring smooth performance under growing demands and holiday spikes. However, manual analysis of results is time-consuming and often incomplete, overlooking critical factors like latency, resource usage and small deviations, which looks negligible from the quick glance. For example, memory leaks in the Golang program.
In this session, you'll learn how we delegated load test analysis by integrating AI into our workflow. By automating result evaluations, AI analyzes complex datasets, going beyond if-else conditions. Our system now delivers actionable insights, identifying issues like memory leaks and CPU inefficiencies, along with improvement suggestions based on benchmarks.
Discover how this AI-powered approach helps us meet resilience and performance goals, scaling from average load to 4x peak demand within minutes, while maintaining p95 under 150 milliseconds.
Usage of default Golang templating in complex report generation.
At Delivery Hero we have a [Reliability Manifesto](https://tech.deliveryhero.com/our-reliability-manifesto/) and in section *R-9* It states that all services should be load tested, be able to process at least 4x load rump-up and should be run on regular bases.
During Load Testing we generate a huge amount of monitoring data in different systems: DataDog APM, Prometheus and AWS Cloudwatch.
All this data should be aggregated in one presentable report.
The problem we want to solve:
- How to abstract reports from different metrics sources and different data they provide.
- How to give a flexible and simple instrument to generate templates automatically.
- How to make the lives of our engineers easier.
For instance, in my tribe we have dozens of services, in Logistics we have hundreds of services, In Delivery Hero overall we have thousands of services.
For each of them we need to run load tests on regular bases.
And for each of them we need to generate reports.
The problem of running tests automatically is easy to solve. But how to solve the problem with reporting?
We have an answer, because we designed a tool which helps us to abstract reports from metrics data sources and at the same time keep flexibility of calling different sources with different queries.
You provide a template in a format of Golang templates
{ { ddResult := queryDD .StartTime .EndTime “query” } }
The maximum value of the metric is { { findMax ddResult } }
Or
{ { promResult := queryProm .StartTime .EndTime “query” } }
{ { drawProm promResult } }
And as the output you receive a generated PDF with either text “The maximum value of the metric is XX” or drawn graph from Prometheus.
All these `queryDD`, `findMax`, `queryProm` and `drawProm` are predefined `template.Funcs` and empower great flexibility of your report.
I will be talking about how to implement these functions, how to call different sources, process data and render outputs.
Surviving Panics, Fatal Errors, and Crashes: Lessons from the Trenches
Ever found yourself in the midst of chaos caused by panics or unexpected crashes? Join me as I share our team's journey through the wilderness of debugging and resolving critical issues that threatened the stability of our system. In this engaging presentation, we'll explore the challenges we faced, the lessons we learned, and the strategies we employed to emerge victorious.
With the rollout of a new feature, our system encountered pod crashes triggered by a dreaded "fatal error: concurrent map iteration and map write." What ensued was a month-long saga of investigation, root cause analysis, and relentless pursuit of solutions. Throughout this ordeal, we discovered invaluable insights that transformed our approach to handling errors and ensuring system resilience.
Our arsenal of survival tactics included:
* Logging, logging, and more logging: Amplifying our logs to capture crucial insights into system behavior.
* Strategic placement of panic recovery: Learning the importance of recovering from panics within the goroutine where they occur.
* The art of reading stack traces: Recognizing the significance of dissecting stack traces, even when they initially seem perplexing.
* Uncovering the hidden impact of non-critical functionality: Understanding how seemingly innocuous components can disrupt critical workflows.
* Distinguishing between fatal errors and panics: Recognizing the distinction between different types of errors and their implications.
* Identifying unrecoverable errors in Go: Gaining awareness of the range of errors that cannot be recovered from in the Go programming language.
By sharing our hard-won wisdom and practical insights, we aim to empower fellow developers to navigate similar challenges with confidence and resilience. Whether you're a seasoned engineer or a newcomer to the field, this presentation offers invaluable guidance for building robust, reliable applications that stand the test of time.
Join us as we unravel the mysteries of error handling and equip ourselves with the tools and knowledge to overcome any obstacle that comes our way. Let's transform setbacks into opportunities for growth and emerge stronger together.
Maximizing Scalability with Go and Redis: A Telemetry Processing Journey
At Delivery Hero, we process a staggering 10,000 requests per second globally, particularly in our critical TIER1 flow, where we handle telemetry data primarily from riders' phones. In this session, we'll delve into how we harnessed the power of Go and Redis to handle this high-load, mission-critical system at an incredibly low cost.
Key Points:
Go and Redis as the Perfect Pair: Discover how Go and Redis form the backbone of our telemetry processing infrastructure. Go's concurrency model and performance complement Redis's high availability and resilience, enabling seamless handling of our demanding operations.
Efficient Data Management with Redis: Explore how Redis's versatile features, including sorted lists and key expiration (TTL), enable efficient telemetry data storage, event queue management, and fraud prevention. Learn how Go's native support for Redis interactions streamlines integration and enhances overall system performance.
Scalability Made Simple: Dive into how Go and Redis effortlessly scale to handle our high-volume telemetry flow. With Go's lightweight footprint and Redis's scalability features, such as automatic sharding and replication, we ensure our system remains responsive and reliable, even under peak loads.
Cost-Effectiveness and Resource Optimization: Learn how we achieved cost-effectiveness by leveraging the smallest Redis instances available, strategically deployed across multiple regions. Explore how Go's efficient resource utilization and Redis's pay-as-you-go pricing model contribute to significant cost savings without compromising performance.
Customized Data Flows and Experimentation: Our implementation with Redis has opened doors for various experiments, including different location frequency updates, giving us the flexibility to optimise and innovate based on unique service needs.testing different telemetry processing strategies, empowering continuous optimization and innovation.
How Regex Works: The Secret Sauce Behind Pattern Matching
Ever wondered how regular expressions pull off their magic tricks? Let's demystify the regex engine together! In this fun talk, we'll peek under the hood and see how it all works, using something called Non-deterministic Finite Automaton (NFA), or as some like to call it, state machines. Don't worry, we'll keep things simple and easy to understand.
The regex engine is like a super-powered detective that's really good at finding specific patterns in text. But did you know that behind the scenes, it's using something we all study in universities, called Non-deterministic Finite Automaton (NFA)? It's a bit like a map that helps the regex engine understand and match patterns efficiently.
Imagine trying to solve a really big puzzle. NFA helps by breaking down complex regex patterns into smaller pieces, or states. Then, all we need to do is follow the map, moving from one state to another until we find our match. It's like having a guide to lead us through the maze of text, making pattern matching a breeze.
During our adventure, we'll take a closer look at how the regex engine is put together. We'll explore concepts like backtracking (when it needs to go back and try a different route), greedy quantifiers (how it decides how much to match), and character classes (the different types of characters it's looking for). And remember, we'll explain everything in plain language, so you won't get lost in technical jargon.
But we won't stop at just talking about theory. We're going to dive deeper and take you through the process of building a simple regex matcher right in the presentation. By walking through this hands-on exercise, you'll get a firsthand look at how the regex engine works in action. By the end of our journey, you'll have not only a better understanding of regex but also the practical skills and confidence to use it effectively in your own projects.
Whether you're a beginner or a seasoned coder, this talk will help you unlock the secrets of regex and level up your pattern matching skills. So, join us for a fun and enlightening exploration into the world of regex!
Automating PDF Generation in Go: From Markdown to Reports
Have you ever needed to generate PDFs in Golang, but didn’t want to reinvent the wheel for every report? In this session, we’ll dive into how we automated the generation of load test reports in Go, using a scalable approach that leverages Markdown as the source for document templates.
Rather than writing custom code for each report, we use a Markdown template to describe the structure and content of our document. Our Go processor fills the template with data and charts, before converting it into a polished PDF. We’ll take a look at how we parse Markdown to recognize text emphasis, images, and tables, and how to build the document using simple primitives like cells and text.
Along the way, we’ll share practical insights on handling edge cases, such as converting and embedding images, managing tables that span pages, and verify everything renders as expected. Whether you're looking to automate reports or simply PDF generation, this session will familiarize you with the tools and knowledge to make your life easier, and your reports more professional.
The "Rabbit Hole" of Dependencies in Golang
Why you should avoid dependencies at any cost. Dependencies in Golang can be deceptive. You start with a simple "Hello, World!" - zero dependencies, a small binary. Then, you add fmt, and suddenly, the build size jumps. A few more dependencies later, and you're lost without even realizing it.
But how bad can it get? Let's say you just need to list S3 buckets. Using the AWS SDK? Your binary is now 14MB. But if you write your own lightweight S3 api implementation? Just 8MB. The culprit? Hidden dependencies, bloated libraries, and engineers not questioning what they import.
This talk isn't just about build sizes: it’s about dependency management culture. In large-scale engineering, poorly managed dependencies cause broken updates, circular dependencies, and the dreaded “helpers” folder that turns into a black hole. I'll share examples from Delivery Hero, showing how small decisions in package design can lead to massive tech debt.
Let’s dig into Golang’s dependency ecosystem, identify the pitfalls, and take a look at how to make better engineering choices without getting sucked into the dependency black hole.
INIT 2025 Dev Conference Sessionize Event Upcoming
Conf42.com Golang 2025
The "Rabbit Hole" of Dependencies in Golang
DevFest Stockholm 2025 Sessionize Event
Norfolk Developers Conference 2025 Sessionize Event
DevFest Berlin 2024
Maximizing Scalability with Go and Redis: A Telemetry Processing Journey
DevFest Hamburg 2024 Sessionize Event
WeAreDevelopers Live 2024 (Season 8) Sessionize Event
GopherCon Europe 2024: Summer Edition in Berlin
Lightning talk: Surviving Panics, Fatal Errors, and Crashes: Lessons from the Trenches
Andrii Raikov
Principal Software Engineer at Delivery Hero SE
Berlin, Germany
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top