Do Multi-Agent Systems Actually Deliver? What Agensh, MAST, and Google Reveal

While Agensh demonstrated that scaling agents can increase benchmark success, the performance curve flattened rapidly. On ProgramBench, the mean pass rate climbed from 19.31% with one agent to 28.78% with 128 agents. On Pandoc, the pass rate rose from 33.89% with a single agent to 55.06% across 1,024 agents. However, roughly 17 of those 21 gained points appeared within the first 127 additional agents (~0.134 points per agent), while the subsequent 896 agents added only about 4 points combined (~0.004 points per agent)—roughly a 30-fold drop in marginal utility per added agent. Furthermore, the paper omitted token usage and dollar costs, leaving unanswered how much coordination overhead was burned to achieve those final incremental gains.

Meanwhile, a UC Berkeley paper titled "Why Do Multi-Agent LLM Systems Fail?" by Mert Cemri and colleagues (arXiv 2503.13657) analyzed failure patterns rather than pure scale. Evaluating over 1,600 annotated traces across seven frameworks (with an inter-annotator Cohen’s kappa of 0.88), the researchers created the MAST taxonomy, classifying breakdowns into 14 distinct modes across three broad areas: system design issues (roughly 44%), inter-agent misalignment (about 32%), and task verification (approximately 24%).

Public discussions frequently mischaracterized the MAST paper by claiming multi-agent setups fail between 41% and 86.7% of the time. In reality, that spread reflected individual framework benchmarks—with 86.7% representing performance on the challenging AppWorld benchmark rather than an overall failure rate for multi-agent architectures in general.

Google Research brought compute parity into the equation with "Towards a Science of Scaling Agent Systems." Testing 180 configurations across four benchmarks (Finance-Agent, BrowseComp-Plus, PlanCraft, and Workbench) and three model families, the researchers normalized the reasoning-token budget so single-agent configurations were granted additional steps to match multi-agent token expenditures.

The results depended entirely on task geometry. On Finance-Agent, which decomposes naturally into independent lookups, a centralized multi-agent system outperformed a single agent by 80.9%. Conversely, on PlanCraft—a sequential planning task where subsequent steps depend strictly on prior outputs—every multi-agent variation performed worse, trailing single-agent setups by 39% to 70%. Error propagation was similarly divergent: independent agent setups magnified errors by roughly 17.2 times, whereas a central coordinator limited error amplification to 4.4 times. Google's researchers also identified a saturation point: once a single agent's accuracy reaches approximately 45%, adding agents often yields minimal upside or degrades performance.

Task Structure and Coordination Bottlenecks

Across all three papers, a single pattern emerges: the topology of the problem dictates whether multi-agent systems help or hurt. Tasks with independent subcomponents reward decomposition, while sequential chains suffer when fragmented across multiple agents. Passing state across model boundaries repeatedly invites misalignment, context drift, and compounding mistakes.

Moreover, coordinator-free setups do not eliminate management—they simply relocate it to message buses and context boards where every agent expends tokens reading, claiming, and synchronizing tasks. Evaluating whether a single advanced prompt or a multi-agent pipeline is right for a specific workload often comes down to empirical testing. For teams benchmarking different model families on sequential versus parallel tasks, developers can try top AI models cheaply through one API at https://apixoai.online.

What it means for developers

Before deploying complex multi-agent frameworks into production, engineering teams should evaluate their architectures against empirical constraints:

• Audit the task dependency chain: If a problem requires strict step-by-step reasoning where step five depends directly on steps one through four, keeping execution within a single agent's context window typically outperforms handing steps between multiple subagents.
• Establish budget-matched baselines: Comparing five agents to a single agent that only receives one inference pass is misleading. Always test whether giving a single agent a self-reflection pass or additional reasoning tokens achieves equivalent accuracy at a fraction of the coordination overhead.
• Use failure taxonomies as diagnostic tools: Rather than blindly adding reviewers when an agentic system breaks, use frameworks like MAST to audit whether the issue stems from flawed role specifications, state synchronization failures, or inadequate task verification.
• Keep coordinators for risk mitigation: Despite the appeal of fully decentralized swarms, centralized coordinators substantially dampen error amplification (holding it to 4.4x versus 17.2x for uncoordinated agents), making them far safer for sensitive production workloads.

Source: More Agents, Smaller Gains: What Agensh, MAST and the Google Study Say About Your Orchestrator — Towards AI. Written by the Apixo team from that report.