Most teams working with AI eventually learn that the model isn’t the thing slowing them down, the vector search layer is. RAG systems, semantic search, and recommendation engines all rely on fast embedding lookups, but CPU-based vector search makes indexing slow, filtering expensive, and query latency unpredictable. Throwing more CPUs at the problem only helps a little, and it still doesn’t feel “real-time,” especially for interactive AI apps.

With OpenSearch 3.x, GPU support changes that dynamic. By pushing index builds and vector scoring to the GPU, teams can finally speed up the heaviest parts of retrieval. In this talk, we’ll share how one production workload went from sluggish, CPU-bound vector queries to fast semantic retrieval by enabling GPU acceleration and tuning hybrid sparse+dense search.

By the end of the session, you’ll have a clear picture of where vector search becomes the bottleneck, how GPUs actually fix it, and what it takes to benchmark and deploy GPU-accelerated retrieval with OpenSearch for real AI use cases.