Study Finds AI Agent Teams Drive Up Token Costs With Minimal Output Gains
New evaluations show AI agent teams cost up to 5.1x more than solo models while delivering almost no quality improvement, raising questions about multi-agent efficiency.
Deploying multi-agent architectures has become a popular strategy for tackling complex programming and reasoning problems. However, recent empirical findings suggest that deploying swarms of AI agents often leads to massive token consumption with negligible performance improvements over a single well-configured model.
Evaluations conducted by Vals AI on the "Vibe Code Bench" revealed that agent teams rarely justify their added expense. Testing GPT-6 Sol and Claude Opus 5.5 in both individual and collaborative setups at medium and maximum reasoning effort levels, the researchers discovered that multi-agent teams cost between 1.8x and 5.1x more than single agents. Despite the steep expense, out of four direct comparisons, only a single scenario demonstrated a statistically meaningful improvement: GPT-6 Sol operating at medium reasoning, where a team achieved a 7.3-point bump. When both models ran at maximum reasoning effort, collaborative setups offered no real benefit over solo configurations.
Diminishing returns in scaling agent counts
Similar ceilings appeared in evaluations conducted by Anthropic using Opus 5.5. In their experiments, expanding the number of agents yielded rapidly diminishing returns on final output quality. While larger clusters reached baseline milestones faster, scaling from 10 to 100 agents produced only marginal score increases after 24 hours.
On a knowledge base benchmark, Opus 5.5 progressed from 0.53 with one agent to 0.70 with 10 agents, 0.71 with 30 agents, and 0.74 with 100 agents. On a Lean Theorem Proving evaluation, performance jumped from 0.39 with a single agent to 0.66 with 10 agents, plateaued at 0.66 with 30 agents, and finished at 0.68 with 100 agents. Additional ProgramBench evaluations demonstrated that any gains in execution speed came at the expense of heavy token overhead. When testing Fable 5.1 on Lean theorem proving, the model showed stronger quality improvements beyond 10 agents but still trailed Opus 5.5 overall. On the knowledge base test, Fable 5.1's benchmark score actually decreased when scaling from 30 to 100 agents.
The coordination tax and task limitations
The consensus that agent clusters prioritize velocity over capability was also addressed by OpenAI researcher Noam Brown during an appearance on the Dwarkesh Podcast. Brown explained that multi-agent systems primarily buy speed rather than improved quality. In observed tests, four agents completed tasks twice as fast while doubling the cost. While the trend persisted up to 16 agents, efficiency degraded slightly as team sizes expanded.
Brown noted that task structure determines viability: workflows such as mathematics and web research parallelize effectively, whereas tasks like novel writing do not. He pointed out that assigning 10,000 agents to write a novel is as counterproductive as assigning 10,000 humans to the same draft. Scaling to massive agent counts remains mostly untested simply because the token costs are prohibitive. OpenAI developer Eric Provencher also cautioned against deploying agent swarms, describing the phenomenon where collaboration collapses as a "coordination tax" that frequently wastes financial resources.
What it means for developers
These findings urge developers to rethink whether multi-agent systems are necessary for their production pipelines. Before spending budgets on complex coordination frameworks, engineering teams should evaluate whether a single frontier model running with elevated reasoning compute can achieve the same or better results at a fraction of the cost.
Rather than assuming that more agents automatically lead to superior code or answers, teams should measure token spend against actual quality deltas. For those looking to benchmark single-agent workflows against multi-agent setups across different model families, developers can try top AI models cheaply through one API at https://apixoai.online. If a task cannot be parallelized cleanly—or if a model is already functioning at maximum reasoning effort—relying on a solo agent remains the more cost-effective architecture.
Source: AI agent teams waste massive tokens for barely measurable quality gains, research finds — The Decoder. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
