Khang Nguyen Hoang

We Deleted Most of Our Agents. Everything Got Faster

The default advice is to add agents: specialised roles, an orchestrator, handoffs between them. We built that. It was a genuinely useful way to explore the problem space, and under real production traffic it was slow and expensive.

When an agent system is slow, the instinct is to reach for a faster model. That was the wrong lever. Our latency wasn't dominated by inference — it was dominated by round trips. Every handoff re-serialised context, every planner turn burned tokens producing nothing a user would ever see, and the overhead compounded per hop. It's the N+1 query problem wearing a different hat: hundreds of small calls where one would do.

So we started deleting. Collapsing to a single agent holding multiple toolkits cut cost per resolved task by roughly two-thirds and latency by about half — same model, no infrastructure change. A caching layer over the repeated-question tail took a further ~30% off the already-reduced figures, on the same principle: the cheapest model call is the one you never make.

I'll show what we measured at each hop to identify round trips as the real cost driver, what broke when we consolidated — tool-selection accuracy, eval attribution, the loss of per-subtask model tiering — and how we handled each. The patterns for managing large toolkits are already known: retrieval over tool definitions, hierarchical grouping, progressive disclosure. What isn't documented is which ones survive production, and where each quietly stops working. Figures are directional and normalised. Includes a side-by-side trace replay of both architectures on an identical task.