In the world of distributed computing, your data is often served through API requests. There may be several systems depending on a single dataset. For datasets that are relatively small (in the order of a few GBs), one way to have faster data access is by using a local in-memory cache. Because the dataset is relatively small, using distributed systems such as Redis/EVCache might be overkill, and the risk of adding a new integration outweighs the reward of faster data access. This is where the services opt for an in-memory local cache.

The JVM ecosystem, in particular, features several highly efficient caching libraries, including Ehcache, Caffeine, and Guava cache, among others. However, there are challenges with maintaining a local in-memory cache. Traditional approaches force uncomfortable trade-offs: cold starts, individual key invalidation, TTLs causing cache stampedes; full reloads causing memory spikes, GC pauses, and service disruptions; and incremental updates require complex change-tracking infrastructure. For gigabyte-scale datasets accessed millions of times per second, these approaches break down—yet this is precisely where in-memory caching matters most.

This talk introduces a different paradigm: treating datasets as versioned, immutable snapshots with delta-based distribution—essentially applying Git's model to in-memory data. We'll explore Hollow, an open-source library by Netflix that implements this pattern, enabling applications to efficiently distribute and update multi-gigabyte datasets with minimal memory overhead, zero downtime, and no GC pressure. Through real-world examples, you'll learn when versioned dataset distribution makes sense, how it works under the hood, and how to apply this emerging pattern to solve persistent caching challenges in your own systems—regardless of whether you're running microservices, monoliths, or anything in between.

Target audience: Backend engineers, Technical leaders, Software Architects
First public delivery: Fremont ACM Chapter
Session duration: 25 mins
Slide deck: https://vichu.github.io/hollowdemo