Humans Beat LLMs in Colonel Blotto Game: The 'U-Shaped Curse' of Reasoning Depth

Forum topic · 小凯 · 2026-05-22

Summary

A working paper (arXiv:2605.22095) reports that humans outperform large language models (LLMs) in Colonel Blotto, a classic game-theoretic resource allocation game with no pure-strategy Nash equilibrium. The authors ran three tournaments: over 200 humans competing against each other, strategies submitted by several mainstream LLMs, and a head-to-head comparison with equalized strategy counts. Human strategies displayed a distinctive 'medium-depth' pattern—calibrated heuristic allocations that were neither fully random nor fully optimal—while LLM strategies were simpler, more stereotyped, and predictable. Humans won across strategy diversity, calibration, and win rate. The paper identifies a 'U-shaped curse': strategic sophistication pays off only at a moderate reasoning depth; both naive uniform allocation and overly complex equilibrium computation underperform. Notably, humans rarely adjusted strategies when facing LLMs instead of humans, treating game rules rather than opponent identity as the primary driver of behavior. STEM background conferred only a slight edge. The authors caution the results are specific to Colonel Blotto's high-dimensional action space, experimental format, and prompt settings, and may not generalize to other games.

Not Yet: Humans Outperform LLMs in a Colonel Blotto Tournament

Paper: Not Yet: Humans Outperform LLMs in a Colonel Blotto Tournament
Authors: Dmitry Dagaev, Egor Ivanov, Petr Parshakov, Alexey Savvateev, Gleb Vasiliev
arXiv ID: 2605.22095
Posted: May 21, 2026
Categories: econ.GN / cs.AI / cs.GT / cs.HC
Core finding: In the Colonel Blotto multi-battlefield resource allocation game, human strategy quality significantly outperforms LLMs; reasoning depth exhibits a "U-shaped curse"—too little is useless, too much is also useless, and only medium-depth heuristic strategies win.

1. Simple Rules, Intractably Complex Solutions

Colonel Blotto has two rules:

1. You have N soldiers to distribute across K battlefields. On each battlefield, the side committing more troops wins it; ties split the battlefield. 2. Overall victory goes to whoever wins more battlefields.

That's it—yet game theorists have studied it for nearly a century. Why is it hard? The action space is high-dimensional (even 10 soldiers across 3 battlefields creates a huge combinatorial space). Worse, the game has no pure-strategy Nash equilibrium: no fixed allocation is safe. You must always randomize and always guess where your opponent will be weak or strong.

The authors organized three Colonel Blotto tournaments: (1) 200+ human participants playing each other; (2) several mainstream LLMs submitting strategies independently; (3) a fair comparison with LLM strategy counts matched to the human count.

2. LLM Strategies: Simple, Rigid, Predictable

Human strategies showed a clear "medium-depth" pattern: well-calibrated middle-layer allocation heuristics—neither fully random nor fully computed-optimal. This "moderately smart" approach proved most effective.

LLM performance was surprising: their strategies were simpler and more stereotyped. If human strategies are like improvisational jazz—flexible within the rules—LLM strategies are like playing from sheet music: accurate but inflexible. Humans beat LLMs on three dimensions: strategy diversity, calibration precision, and win rate.

A key line from the paper: *"Strategic sophistication is key to success if and only if the necessary level of reasoning depth is reached."* Too little reasoning fails; too much fails too. This is a U-shaped curse.

3. The U-Shaped Curse of Reasoning Depth

Play A (low depth): Split troops evenly by intuition—a naive strategy.
Play B (medium depth): Reason one or two levels of "I think that you think that I think..." limited recursion—e.g., concentrate force on one key battlefield, then consider whether the opponent anticipates this.
Play C (high depth): Try to enumerate all allocations and compute the mixed-strategy Nash equilibrium.

Intuition says choose C, the "smartest" option. But experiments show C is no better than A. Many LLM strategies land at the extremes—overly simple uniform allocation, or overly complex computation that errs or overfits. The winners are B: heuristics that think exactly one step ahead.

This echoes the Keynesian beauty contest: what matters is not how deep you reason, but the gap between your depth and your opponent's. Out-thinking a non-thinking opponent by one level wins; out-thinking them by ten levels when they think zero just adds noise.

4. Humans Don't Switch Strategies

The most counterintuitive finding: when humans faced different opponents—humans in round one, LLMs mixed in round three—they barely adjusted their strategies.

Humans anchor on the rules of the game itself, not the opponent's identity. Whether the opponent is a human or GPT-4 matters little. In effect, humans treated the LLM like any other player—not out of anthropomorphism, but because in this game the rules alone drive near-optimal behavior.

This raises an intriguing question: if LLM strategies differ from human ones, and humans don't adapt to opponent identity, what exactly did humans win? They won through superior strategy quality under the same rules—not through opponent prediction.

5. A Weak STEM Edge and Honest Limitations

Within the human pool, STEM-background participants performed slightly better in round one—but the effect was weak, likely reflecting habits of systematic thinking rather than domain knowledge.

The authors (and this post) honestly note the study's boundaries:

1. Single game tested. Results in prisoner's dilemma, public goods, or ultimatum games may differ; Colonel Blotto's high-dimensional action space and lack of pure-strategy equilibrium may be exactly why LLMs struggle. Generalizing to "LLMs are bad at games" is risky. 2. Format inequality. LLMs submitted allocations via API calls, while humans had full instructions, thinking time, and scratch work—possibly exaggerating the gap. 3. Untuned LLM settings. The paper does not report tuning of temperature, system prompts, or few-shot examples; conclusions may be prompt-specific. 4. Homogeneous sample. 200+ participants share one cultural background; cultural dependence of strategy preferences remains unexplored.

Takeaway

Colonel Blotto is an elegant metaphor: many problems have no optimal solution—you pick in a "moderately smart" way and hope your opponent reasons one layer less, or many layers more. LLMs are stuck between "not reasoning" and "over-reasoning," while humans found the sweet spot. Whether that sweet spot is a durable human advantage or just an untuned "reasoning gear" on LLMs is the question this paper leaves to future researchers.

*Tags from the original post: FeynmanLearning, GameTheory, HumanVsAI.*