A client deploying an AI retrieval system quickly learned that the model wasn’t the slow part; vector search was. As embeddings and traffic increased, CPU-bound retrieval made indexing slow and latency unpredictable, breaking the “real-time” experience that semantic search and RAG workloads needed.

In this talk, we’ll show how GPU-accelerated vector search in OpenSearch closed that gap. We’ll compare CPU vs GPU behavior for ingestion and queries, explain where GPUs make the biggest difference, and share the deployment lessons that made performance stable enough for production.

You’ll learn how to:
1. Identify when vector search becomes the bottleneck
2. Benchmark CPU vs GPU retrieval performance
3. Use GPU offloading to speed up hybrid semantic search workflows

If you’re building semantic search or RAG pipelines, this talk gives you practical performance lessons instead of theory.