As more companies are starting to build internal agents that sit on top of their tools, data, and workflows. What often starts as an assistant connected to a few systems quickly grows into something used for research, recruiting, sales, product, engineering, and other functions across the company.

As these systems grow, the agent itself starts accumulating more tools, skills, memory, permissions, and organizational context. At that point, an architectural decision starts to emerge: do you continue with one generalist agent that understands the organization and delegates work through sub-agents when needed, or split the system into specialized agents, each built around a particular function?

We encountered this while building our own internal agent infrastructure around Hermes. We chose the generalist approach: a single self-hosted Hermes instance used across the company through Slack, connected to our internal tools and skills, and capable of spawning sub-agents for specific tasks. Around the same time, a friend building a similar internal agent system at another company had taken the opposite approach, running roughly 18 separate specialized agents for different functions.

Both architectures can accomplish many of the same tasks, but they organize intelligence very differently. With a generalist, context and capabilities are shared and specialization happens at task time. With specialized agents, context, memory, tools, and behavior can be shaped around a role before a task ever arrives. That changes how much an agent needs to figure out on every run, what information it carries, how work gets delegated, and ultimately how predictable the system becomes as it scales.

This talk breaks down what actually happens underneath both architectures and the trade-offs that emerge around context and memory, tool and model selection, orchestration, performance, and operational complexity. Using these two real-world deployments as a case study, we’ll look at where each architecture works well, where it begins to break down, and how to decide when a generalist with sub-agents is enough versus when specialized agents become the better abstraction.