Session

Reduce AI Agent Costs with Semantic Tool Selection

Your AI agent has 29 tools. On every single call, all 29 tool descriptions get serialized into the context window, whether the user asks about weather or hotel bookings. That is thousands of tokens wasted per query, and the LLM still picks the wrong tool 15% of the time. The dual problem: As agents scale beyond 10-15 tools, two things break simultaneously. First, the LLM struggles to select the correct tool from a crowded context, leading to tool hallucination where it calls tools that do not exist or picks the wrong one. Second, every tool description consumes tokens on every call, inflating costs linearly with tool count. I will cover why tool descriptions are the hidden cost driver in agent architectures, how semantic tool selection uses FAISS + SentenceTransformers to filter tools, three implementation approaches (basic filtering, threshold-based, and hybrid), dynamic tool swapping while preserving conversation memory, and a live comparison of all-tools vs semantic selection on the same queries. You'll walk away with: • Working semantic tool selection implementation with FAISS • Tool registry pattern with embeddings and metadata • Memory preservation across dynamic tool swaps • Token cost calculation and comparison methodology • Open source code for a 29-tool travel agent system Most agent optimization talks focus on prompt engineering or model selection. This addresses the overlooked architectural problem of tool management at scale. You will see exact token counts, error rates, and cost comparisons, not theoretical improvements but measured results from a working system.

Outline: • The Dual Problem • Solution Architecture • Live Implementation • Production Pattern • Advanced Patterns

Elizabeth Fuentes Leone

Developer Advocate

San Francisco, California, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top