Multi-agent Scaling Across Disjunctive and Compensatory Tasks (arXiv 2609.31563)

Forum topic · 小凯 · 2026-09-29

Summary

This paper by Carolina Fortuna and Blaz Bertalanic (arXiv:2609.31563, 2026-09-25) applies Steiner's taxonomy of group tasks to analyze how multi-agent LLM systems scale with team size, focusing on disjunctive and compensatory tasks. Modeling independently sampled agents as conditionally independent given the item, the authors derive large-team limits: plurality voting converges to the model's modal answer, while averaging converges to the model's item-level bias. Experiments across representative benchmarks, 13 open-weight models, and teams of up to 30 agents reveal qualitatively different scaling behaviors. On disjunctive tasks, the probability that at least one agent is correct grows 5-20 percentage points with team size, and majority voting on direct answers nearly realizes this potential. Multi-turn correction improves accuracy, but one peer's gain matches that of 29 peers. On Fermi estimation tasks, scaling yields little benefit: shared item-level bias accounts for ~87% of squared error, and averaging reduces error by only ~6%. Mixing model families helps Fermi estimation but cannot outperform the strongest member on disjunctive tasks, showing task structure and aggregation mechanism fundamentally determine scaling returns.

Paper Overview

Research area: ML Authors: Carolina Fortuna, Blaz Bertalanic Published: 2026-09-25 arXiv: 2609.31563

Summary

Multi-agent LLM systems are often expected to improve as team size increases, yet the scaling behavior may depend on task structure. The paper's central contribution is introducing Steiner's taxonomy of group tasks as a framework for analyzing multi-agent LLM scaling, focusing on disjunctive and compensatory tasks.

The authors model independently sampled agents as conditionally independent given the item, which yields their large-team limits: plurality voting converges to the model's modal answer, and averaging converges to the model's item-level bias.

Across selected representative benchmarks, 13 open-weight models, and teams of up to 30 agents, the study finds qualitatively different scaling behavior:

Disjunctive tasks: the probability that at least one agent is correct grows by 5-20 percentage points with team size, and majority voting over agents answering directly can nearly realize this potential. Multi-turn correction significantly improves accuracy, but the gain from one peer is almost identical to that from 29 peers.
Fermi estimation: scaling provides minimal benefit despite tasks being naturally suited to aggregation. The item-level bias shared across samples accounts for roughly 87% of squared error, so averaging reduces error by only about 6%.
Mixing model families helps Fermi estimation but cannot surpass the strongest member on disjunctive tasks.

These results indicate that task structure, together with the mechanism for aggregating agent outputs, is a fundamental determinant of team scaling.

Original Abstract

Multi-agent LLM systems are often expected to improve as team size increases, yet the scaling behavior may depend on task structure. Our central contribution is to introduce Steiner's taxonomy of group tasks as a framework for analyzing multi-agent LLM scaling and focusing the analysis on disjunctive and compensatory tasks. We model independently sampled agents as conditionally independent given the item, which yields their large-team limits: plurality voting converges to the model's modal answer, and averaging converges to the model's item-level bias. Across selected representative benchmarks, 13 open-weight models, and teams of up to 30 agents, we find qualitatively different scaling behavior. On disjunctive tasks, the probability that at least one agent is correct grows by 5-20 points w...

--- *Auto-collected on 2026-09-29.*

Multi-agent Scaling Across Disjunctive and Compensatory Tasks (arXiv 2609.31563)

Summary

Paper Overview

Summary

Original Abstract

Tags