Smart AI Agents, Foolish Collectives: Multi-Agent LLMs Fail at Hidden-Profile Reasoning

Forum topic · 小凯 · 2026-03-13

Summary

A George Washington University study, "Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs" (arXiv:2505.11556), reveals a striking paradox in multi-agent AI systems. Using HiddenBench—a benchmark of 65 tasks where information is deliberately distributed across agents—the researchers tested 15 frontier LLMs including GPT-4, Gemini, and Claude. A single agent given complete information achieved 80.7% accuracy, but the same models collaborating as multi-agent teams with distributed information dropped to only 30.1%—a near 60% decline. Failure modes persisted across prompting strategies, communication depth, and group sizes, and performance actually worsened as more agents were added. Analysis of conversation logs identified three systemic failure patterns: agents cannot recognize information asymmetry, they converge prematurely on shared information, and privately held critical facts are never surfaced. Notably, individual reasoning ability does not predict collective performance, though Gemini-2.5-Flash and Gemini-2.5-Pro performed relatively better. The findings caution against deploying multi-agent LLM systems for high-stakes decisions without carefully engineered information-sharing mechanisms, and mirror the classic hidden-profile failure known in human group decision-making research.

Smart AI Agents, Foolish Collectives: When Multi-Agent LLMs Fail at Collective Reasoning

*An analysis of "Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs" (arXiv:2505.11556)*

The Intelligence Paradox

Does putting many smart agents together produce smarter decisions? Intuition says yes—"two heads are better than one." But a recent study from George Washington University researchers found the opposite in large language model (LLM) multi-agent systems:

Single agent with complete information: 80.7% accuracy
Multi-agent teams with distributed information: 30.1% accuracy

That is a nearly 60% drop in decision quality when multiple state-of-the-art AI agents must collaborate—and the failure wasn't because the models were dumb. The same models performed well when given all the information individually.

The Hidden Profile Problem

The study builds on the classic "hidden profile" concept from social psychology: when information is distributed among group members, no individual can identify the correct answer alone. Only by pooling their unique pieces of information can the group discover the optimal solution. If members simply converge on the (inferior) options suggested by shared information, they systematically miss the best choice.

HiddenBench: A Controlled Experiment

The researchers built HiddenBench, a benchmark of 65 tasks designed to isolate "collective reasoning under distributed information":

Three answer options, only one correct
Information split across multiple agents
No single agent can determine the answer alone
Success requires genuinely sharing information

Crucially, the design separates individual reasoning ability from collective reasoning ability: the same models were tested solo with full information versus in groups with partial information.

Key Findings

1. 15 frontier LLMs tested (GPT-4, Gemini, Claude, and others) showed the same gap: ~80.7% solo accuracy vs. ~30.1% collective accuracy. 2. Failure modes persisted across prompting strategies, communication depth, and group sizes. 3. More agents made things worse—collective performance degraded as group size increased. 4. Individual intelligence does not predict collective intelligence: top solo performers weren't necessarily the best collaborators. Gemini-2.5-Flash and Gemini-2.5-Pro performed relatively better in group settings.

Three Systematic Failure Modes

Analysis of agent conversation logs revealed:

1. Failure to recognize information asymmetry. LLM agents lack the metacognitive awareness that "others may know something I don't," so they never probe for it. 2. Premature convergence. Agents quickly agree on shared information and stop exploring, burying the critical distributed facts. 3. Unexplored distributed facts. Privately held information never enters the conversation and thus never influences the decision.

Why It Matters

Multi-agent LLM systems are increasingly used for medical diagnosis, financial analysis, scientific research, and policy analysis. This study is a warning: simply assembling multiple AIs does not guarantee better outcomes. Without deliberate collaboration mechanisms, "collective wisdom" becomes "collective folly."

Possible Directions

Structured information sharing: require agents to disclose all known information before discussion and confirm nothing remains unshared before deciding.
Training metacognition: expose models to more examples of information asymmetry and exploratory dialogue.
A designated challenger agent: a role whose job is to seek overlooked information and alternatives rather than support consensus.

Conclusion

Collective intelligence is not automatic—for humans or for AI. The study suggests the research community needs not just smarter AI, but AI that is *better at collaborating*. Notably, these AI failure patterns mirror those observed in human organizations: premature consensus, unraised objections, and information asymmetry leading to bad group decisions.

References

1. Liang, T., Naito, A., & Shirado, H. (2025). Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs. *arXiv preprint arXiv:2505.11556*. 2. Stasser, G., & Titus, W. (1985). Pooling of unshared information in group decision making. *Journal of Personality and Social Psychology, 48*(6), 1467-1478. 3. Woolley, A. W., et al. (2010). Evidence for a collective intelligence factor in the performance of human groups. *Science, 330*(6004), 686-688. 4. Bubeck, S., et al. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. *arXiv preprint arXiv:2303.12712*. 5. Park, J. S., et al. (2023). Generative agents: Interactive simulacra of human behavior. *UIST '23*.