More AI Agents Did Not Make the App Better

Multi-agent setups are the default pitch for AI coding tools this year. A lead agent splits a job, hands the pieces to subagents that work in parallel, and merges the results. It sounds like a sensible way to build something bigger, faster. Vals AI, an evaluation company, decided to measure what the extra agents actually buy.

On October 9, 2026, Vals AI published the results. The team used Vibe Code Bench, its benchmark for building complete web apps from a product specification. Each app was deployed and then graded by browser agents that clicked through its workflows. The test covered two models, OpenAI's GPT 6 Sol and Anthropic's Claude Opus 5.5, at medium and maximum reasoning effort. Each model ran once as a single agent and once as a lead agent with up to five subagents. The test used 50 apps.

Teams cost between 1.8 and 5.1 times as much as single agents. In only one of the four comparisons did the team score significantly higher. That was GPT 6 Sol at medium effort. Its team scored 84.9 percent against 77.6 percent for the single agent, and cost $3.07 per app against $1.22. The 7.3-point gain held up statistically. The other three comparisons moved the score by between minus 0.3 and plus 3.4 points, which the study could not tell apart from chance.

The most expensive setup was not the best value. Claude Opus 5.5's team at maximum effort scored 93.2 percent at $122 per app on average. Its single agent at the same effort scored 89.8 percent for $23.77. At medium effort, the single Opus agent scored 91.5 percent for $4.08 per app, and the team scored 91.2 percent for $9.10.

Why teams cost so much

The study points to one mechanism. Each subagent holds its own context, including the spec and its working notes, and that context is sent again on every model call. At maximum effort, Opus's subagents read a median of 224 million cached tokens per app. The single agent read 55 million. Coordination adds more work on top, because the lead has to split the job, check the pieces, and reconcile them.

The study also found that more reasoning effort often helped more than a team did. For Sol, the single agent at maximum effort scored 89.0 percent at $4.82 per app. That was 4.1 points above the medium-effort team, a gap the study could not separate from chance, and the maximum-effort team cost $8.54 per app. Vals' summary says higher effort is an alternative worth weighing against a team, and I think that is the right way to read the numbers.

What this means if you are building a product

If you are a solo founder or a small team, cost is your margin. Suppose you use an AI coding tool to ship features every week. A setup that costs five times as much per feature does not give you five times the output. In this study it bought almost nothing in most comparisons.

I would measure three things for your own work: cost per shipped feature, the number of attempts before a feature works, and time until the test suite passes. Then try a single agent at a higher effort setting before you try a team. Add subagents only to tasks that split into parts that can truly run side by side.

Keep the limits of the study in mind. It used one benchmark of full-stack web apps. Each setup ran once, so the intervals reflect variation across apps rather than repeated runs. Vals says the results may not carry over to other kinds of work. Treat the numbers as a strong signal for this type of task, and run your own tests on yours.

Sources

• Vals AI, "Do Agent Teams Pay Off? A Case Study on Vibe Code Bench," October 9, 2026: https://www.vals.ai/blogs/multi-agent-vibe-code-bench