A lot of coding agents find their way around a codebase with text search and a shell. Meanwhile the IDE sitting right next to them already knows every caller of every method, and it can rename a symbol across a whole solution without touching the wrong thing. Some tools can now hand that to an agent over MCP, so I wanted to know if it actually helps, and by how much. To be honest, I don't know yet.

I'm running the same .NET tasks three ways. One agent only gets file search and the shell. Another gets IDE tools over MCP. The third gets an open-source setup built on a language server, so anyone can reproduce it without a license. The tasks are the kind text search struggles with, like a rename across a solution, a signature change with a lot of callers, and a bug that only shows up in one test. For every run I'm tracking tokens, time, whether it worked, and how often the build broke along the way.

By the end you'll know which kinds of tasks are worth wiring up semantic tools for, and how to set them up for your own agent. You'll also get the harness, so you can run the same comparison on your own code.