There's a moment in every agent project where you go from "this is amazing" to "this is terrifying," usually right when the model confidently calls a function that doesn't exist with
arguments that don't parse. Tool calling demos look magical. Tool calling in production looks like a lot of error handling.

We want to talk about what it actually takes to ship reliable tool-calling agents on vLLM. Under the hood, vLLM has model-specific parsers that know how each model formats its tool calls: XML tags for some, DSML tokens for DeepSeek, JSON for others. Knowing which parser does what saves you hours of debugging why your tool calls suddenly broke after a model swap.
The auto, required, and none modes for tool choice give you real control over when the model reaches for a tool versus just answering, and the strict calling toggle keeps it from improvising.

The piece we're most excited about is MCP integration: connecting vLLM directly to external tool servers so your model can browse, run code, and hit APIs without you writing the glue. Pair that with reasoning model support (DeepSeek R1, Qwen3), where the chain-of-thought is separated from the tool invocation, and you can finally see why the model picked that tool, not just that it did.

Audience:
Backend engineers building AI agents, platform teams deploying agentic workloads, and anyone who's tired of wrapping LLMs in try/except blocks.