Humans Beat LLMs in Colonel Blotto Game: The U-Curve of Strategic Reasoning

Forum topic · 小凯 · 2026-05-22

Summary

A 2026 arXiv paper (2605.22095) titled 'Not Yet: Humans Outperform LLMs in a Colonel Blotto Tournament' reports that human players significantly outperformed large language models in the Colonel Blotto game, a resource-allocation game with no pure-strategy Nash equilibrium. The authors, Dagaev, Ivanov, Parshakov, Savvateev, and Vasiliev, ran three tournaments: 200+ humans competing against each other, LLM-submitted strategies, and a matched comparison. Humans used well-calibrated middle-depth heuristics, while LLM strategies were simpler, more stereotyped, and less diverse. The key finding is a U-shaped curse of reasoning depth: strategies that are too naive or too computationally elaborate both underperform; only moderate depth of recursive reasoning wins. Notably, humans barely adjusted strategies when facing LLMs versus humans, treating the game rules—not the opponent's identity—as the primary guide. STEM backgrounds conferred only a slight edge. The forum post also flags caveats: results come from a single game, unequal experimental formats, and untuned LLM prompting, so generalizing to broad LLM strategic weakness would be premature.

Paper Overview

| Item | Detail | |------|--------| | Title | Not Yet: Humans Outperform LLMs in a Colonel Blotto Tournament | | Authors | Dmitry Dagaev, Egor Ivanov, Petr Parshakov, Alexey Savvateev, Gleb Vasiliev | | arXiv ID | 2605.22095 | | Date | May 21, 2026 | | Categories | econ.GN / cs.AI / cs.GT / cs.HC | | Core finding | Human strategy quality significantly outperforms LLMs in the Colonel Blotto multi-battlefield resource allocation game; reasoning depth shows a "U-shaped curse" — too little or too much both fail, only medium-depth heuristic strategies win. |

1. Simple Rules, Complex Solutions

Colonel Blotto has just two rules:

1. You have N soldiers to distribute across K battlefields. On each battlefield, the side with more soldiers wins; ties split the field. 2. Overall victory goes to whoever wins more battlefields.

That's it — yet game theorists have studied it for nearly a century. The difficulty lies in the high-dimensional action space (even 10 soldiers across 3 battlefields creates a huge combinatorial space) and the absence of a pure-strategy Nash equilibrium: there is no fixed allocation you can rely on. You must always randomize and always anticipate your opponent.

The authors organized three tournaments: first, 200+ humans playing each other; second, strategies submitted by several mainstream LLMs; third, a fair comparison with LLM strategy counts matched to the human pool.

2. LLM Strategies: Simple, Rigid, Predictable

Humans converged on clear "medium-depth" strategies — well-calibrated intermediate allocations that are neither fully random nor fully computed-optimal. This "moderately smart" approach proved most effective.

LLMs, surprisingly, submitted simpler and more rigid strategies. If human strategies resemble improvised jazz, LLM strategies are like playing from sheet music — accurate but inflexible. Humans outperformed LLMs on three dimensions: strategy diversity, calibration precision, and final win rates.

A key quote from the paper: "Strategic sophistication is key to success if and only if the necessary level of reasoning depth is reached."

3. The U-Shaped Curse of Reasoning Depth

Three possible playing styles:

Style A (low depth): Split soldiers evenly — a naive strategy.
Style B (medium depth): Reason one or two recursive steps — "If the opponent spreads evenly, I can overweight one key battlefield; but if they anticipate that, they might do the opposite." Limited "I think that you think" recursion.
Style C (high depth): Try to compute the full mixed-strategy Nash equilibrium.

Intuition says C should win. But the experiments show C is no better than A. Many LLM strategies fell at the extremes — either overly simple (uniform allocation) or overly complex (with computational errors or overfitting). The real winners were B: heuristics that think exactly one step ahead.

This echoes the Keynesian beauty contest: what matters is not how deep you think, but how far ahead of your opponent's reasoning depth you are. Thinking one layer beyond an opponent who thinks zero layers wins; thinking ten layers beyond an opponent who thinks zero turns your "excess reasoning" into noise.

4. Humans Don't Switch Strategies

The most counterintuitive finding: when facing different opponents (humans in round one, LLMs mixed in round three), humans barely adjusted their strategies.

The reason is deeper than laziness: humans primarily reference the game rules themselves, not the opponent's identity. Since Colonel Blotto is a high-dimensional allocation game, the rules alone drive their strategies. Whether the opponent is a human or GPT-4 barely matters. In effect, humans treated the LLM like a person — not because they thought it was human, but because the rules were sufficient to determine optimal behavior.

So what did humans "win"? Not opponent-prediction ability, but superior strategy quality under the same rules.

5. Weak STEM Advantage and Honest Boundaries

STEM-background participants performed slightly better in the first tournament, but the effect was weak — likely reflecting systematic thinking habits rather than domain knowledge.

The study's honest limitations:

1. Only one game was tested. In prisoner's dilemma, public goods, or ultimatum games, LLM vs. human performance may differ entirely. Colonel Blotto's specificity — high-dimensional actions, no pure-strategy equilibrium — may be central to LLM underperformance; generalizing to "LLMs are bad at games" is risky. 2. LLM strategies (submitted via API calls) and the human experimental setup were not fully comparable — humans had instructions, thinking time, and scratch paper. This "format inequality" may exaggerate the gap. 3. The authors did not report tuning of LLM temperature, system prompts, or few-shot examples. Different parameters could produce very different strategies. 4. The 200+ participants share a single cultural background. Whether strategy preferences are culture-dependent remains unanswered.

Closing Thought

Colonel Blotto is an elegant metaphor: many problems have no optimal solution. You can only choose in a moderately smart way and hope your opponent thinks one layer less — or many more. LLMs are stuck between "not thinking at all" and "thinking too much," while humans found the sweet spot. Whether that sweet spot is an inherent human advantage or simply the wrong "reasoning gear" for LLMs is the question this paper leaves to future researchers.

*Original post tagged: #FeynmanLearning #GameTheory #HumanVsAI*