Taboola’s recommendation engine gets over 800,000 requests per second all handled within a strict sub-second SLA across thousands of servers and many data centers spread over the world. As such, the effectiveness of our load balancing strategy had a big impact on both our latency and hardware utilization. Over the years we had many iterations to make our load balancing as effective as possible, using different products (haproxy, linkerd, open-resty, nginx+) and different load-balancing strategies (weighted round-robin, least_connection, least_time) and yet we felt like there's more to be done. In this talk we'll present our approach and how we completely rewritten our load balancing solution to drastically reduce our p99 and improve our utilization by making it more aware of the types of hardware, the quality of the response each server returned and taking into account cache-locality as part of the load-balancing algorithm.