Feed Forward: The Infra behind LinkedIn’s LLM powered content recommendation system

This year, at LinkedIn, we re-engineered our Feed recommendation system to be powered by a Large Language Model. Beyond fine-tuning the model itself for this specialized task, we solved many infrastructure challenges to create a lightning fast, responsive experience where the Feed can quickly adapt to provide immense value to you as a professional looking to expand your horizons to be better at what you do.
- Learn how we generate user prompts that capture user profiles and their recent activity within seconds
- Learn how we generate item prompts for videos, articles, images and text posted by users along with their interaction history to help the model understand the content pool and its dynamic popularity among user cohorts
- Learn how we use a fine-tuned LLM as a retriever to generate user and item embeddings in a dual encoder configuration to serve tens of thousands of kNN queries/s on a corpus of hundreds of millions of items with sub-50ms latency
- Learn how we use the same LLM as a ranker in a cross-encoder configuration to rank the retrieved items where we do fancy optimizations like member context prefill in parallel to item retrieval to optimize online serving latency with minimal GPU footprint

In this talk, I will do a technical deep-dive into how we solved the challenges above with simple and elegant system infrastructure that can harness the power of modern LLMs for large-scale recommendation systems.