CooperBench: The Curse of Coordination — Why Two GPT-5 Agents Working Together Perform 50% Worse

Forum topic · 小凯 · 2026-06-20

Summary

Researchers from Stanford University and SAP Labs US introduced CooperBench (arXiv: 2601.13295), the first benchmark specifically designed to test collaboration between AI coding agents. It comprises 652 tasks drawn from real pull requests across 12 open-source repositories in four languages (Python, TypeScript, Go, Rust), requiring two agents to implement potentially conflicting features in parallel on a shared codebase. The key finding is a 'curse of coordination': in cooperative mode, top models like GPT-5 and Claude Sonnet 4.5 achieve only about 25% success, versus roughly 50% when a single agent handles both tasks sequentially — a 50% coordination gap present across every tested model. Failures fall into three categories: expectation failures (42%), commitment failures (32%), and communication failures (26%). Notably, giving agents a communication channel did not improve success rates, despite consuming up to 20% of compute budgets. The paper argues that the bottleneck for AI agents is not task competence but social intelligence, and calls for research into theory of mind, commitment maintenance, and shared mental models. Rare emergent behaviors — role division, resource division, and negotiation — hint that reinforcement learning on CooperBench could teach agents to coordinate.

Researchers from Stanford University and SAP Labs US have published a counterintuitive finding: when two top-tier AI coding agents (GPT-5, Claude Sonnet 4.5) team up on real codebase tasks, the result is not 1+1=2 but 1+1=0.5. Two agents working together succeed about 50% less often than a single agent completing both tasks alone. The bottleneck is not compute or model capability — it is that AI fundamentally lacks "social" skills. The paper introduces CooperBench, the first benchmark specifically testing AI agent collaboration.

What is CooperBench?

Paper: CooperBench: Why Coding Agents Cannot be Your Teammates Yet
Authors: Arpandeep Khatua, Hao Zhu, Peter Tran, Arya Prabhudesai, Frederic Sadrieh, Johann K. Lieberwirth, Xinkai Yu, Yicheng Fu, Michael J. Ryan, Jiaxin Pei, Diyi Yang
Institutions: Stanford University, SAP Labs US
arXiv: 2601.13295
Website: https://cooperbench.com

Task design: 652 tasks extracted from real PRs across 12 open-source repositories (Python: Flask, Scikit-learn, LlamaIndex, Pandas, Matplotlib; TypeScript: Prisma, Vue; Go: Kubernetes client; Rust: Tokio, Clap). Each task splits a multi-feature PR into two independent but potentially conflicting subtasks. Two agents develop in parallel in isolated sandboxes, sharing a Git repository and an optional messaging channel. Success requires that both patches merge and all tests pass.

The Coordination Gap

| Model | Solo success | Coop success | Coordination gap | |-------|-------------|--------------|------------------| | GPT-5 (OpenHands) | ~50% | 25% | 50% | | Claude Sonnet 4.5 | ~50% | 25% | 50% | | Minimax M2 | ~50% | 25% | 50% | | Gemini 3 Flash | ~50% | 27.76% | ~44% | | Qwen3-30B-A3B | ~15% | ~10% | ~33% |

Every model shows a coordination gap — no exceptions. The gap is largest for top models (strong solo ability, weak collaboration) and for medium-difficulty tasks. Even the leaderboard leader (GPT-5, 27.95% coop success) sits far below the ~50% solo baseline. Allowing Git collaboration barely changes results, showing that code sharing is not the bottleneck — semantic coordination is.

Three Failure Modes

1. Expectation failures (42%): Agents hold wrong assumptions about teammates' plans, observations, and states — e.g., silently assuming an unchanged API format while the teammate changes it. 2. Commitment failures (32%): Agents break their own commitments without updating or notifying teammates when circumstances change. 3. Communication failures (26%): Vague, mistimed, or inaccurate messages; questions ignored; redundant status updates consuming up to 20% of compute budgets.

The Communication Paradox

Giving agents a real-time chat channel did not significantly improve success rates. Communication reduced Git merge conflicts (spatial coordination) but not logical incompatibilities (semantic coordination). Message volume crowded out genuinely critical information, and agents often failed to integrate received information into their plans.

Emergent Coordination Behaviors

In rare cases, agents spontaneously exhibited:

Role division: explicitly partitioning code responsibilities ("I'll add header + octal_str; you add binary_str between them")
Resource division: identifying which files/functions need serial vs. parallel work
Negotiation: proposing designs, questioning test coverage, and agreeing on conditions

These behaviors were not prompted or scaffolded, suggesting that reinforcement learning on CooperBench could train coordination skills.

Why Current AI Cannot Collaborate

1. Lack of theory of mind: agents don't model teammates as systems with their own beliefs, goals, and plans. 2. Lack of a shared mental model: each agent sees only its own task, with no global view of the joint objective. 3. Missing commitment and trust mechanisms: agents don't track promises or proactively announce when they can no longer keep them.

Implications

The authors call for a research shift from individual agent competence toward social intelligence — understanding others, communicating effectively, and coordinating action. Future directions include theory of mind for agents, commitment maintenance mechanisms, shared mental model construction, and RL-trained emergent coordination.

Limitations: the benchmark tests only two agents, synchronous collaboration, and AI-AI teams; asynchronous workflows, human-AI hybrid teams, and long-term trust building remain open.

Bottom line: today's agents can do, but not ask; can write, but not communicate; can work, but not negotiate. GPT-5 can write perfect code yet not know a teammate changed the same function. CooperBench quantifies this harsh reality — and suggests that training social intelligence may be harder than training the next frontier model.