Session
Partitioning with Purpose: Kafka Producer Strategies that Cut Lag ~30%
Hot partitions and uneven key distribution are common—and they quietly cap throughput. This session breaks down how purposeful partitioning (entity‑affinity keys, composite keys, and targeted salting) plus tuned producer configs (batch.size, linger.ms, acks, idempotent producers) reduced consumer lag by ~30% in a high‑throughput pipeline. We’ll align partition strategy with consumer concurrency, discuss sticky vs uniform distribution, and show practical skew detection (per‑partition lag, heatmaps) and mitigation (pre‑partitioning, re‑keying). You’ll leave with a field‑tested checklist to diagnose hotspots, tune batching, and roll out changes safely without thundering herd rebalances.
Ishan Shah
PayPal, Software Engineer
San Francisco, California, United States
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top