One-line summary: Former UK Prime Minister's Office data scientist Liam Wilkinson spent a weekend building 76 MCP tools and threw Claude Opus 4.6 / GPT-5.4 / Gemini 3.1 Pro and four top models into Civilization VI for 23 matches. Claude, playing Portugal, built a nuke and flattened France's Toulouse — and still lost, because it never noticed France accumulating diplomatic victory points. Wilkinson extracted two fatal numbers: active global-state checks account for only 1-2% of actions (the sensorium effect), and only 48-66% of written plans are executed within 10 turns (the knowing-doing gap).
What Happened
On June 28, 2026, IT Home republished a report from QbitAI about the CivBench experiment (posted on Wilkinson's personal blog). Key facts:
- Author: Liam Wilkinson, former UK Prime Minister's Office data scientist (creator of GovBench, where GPT-5 scored 99.26)
- System: 76 MCP tools; AI perceives the game through a text interface (no visuals, only pipe delimiters and hex coordinates)
- Models tested: Claude Opus 4.6, GPT-5.4, Gemini 3.1 Pro, and four total
- Setup: 3 difficulty tiers (Ground Control / Snowflake / Cry Havoc), 23 matches
- Key findings:
- Sensorium effect: AI actively checks global state only 1-2% of the time
- Knowing-doing gap: plan execution rate within 10 turns — Claude Opus 4.6 at just 48.2%, GPT-5.4 at 63.2%, Gemini 3.1 Pro highest at 65.8%
- A decision space of roughly 10^166 possible actions per turn (Go has 10^360 per move, but Go places one stone; Civ VI simultaneously moves dozens of units, picks buildings, sets tech, and conducts diplomacy)
- Multiple concurrent objectives (science, culture, diplomatic, domination, religion victories)
- Long causal chains (build → production → resources → eureka moments → victory progress)
- Incomplete information (you can't see opponents' full tech trees or true intentions)
- Claude Opus 4.6: 48.2% (lowest, under half)
- GPT-5.4: 63.2%
- Gemini 3.1 Pro: 65.8% (best — yet a third of plans still rotted in the diary)
- Perception is an architecture problem, not an intelligence problem. The AI only gets information by actively calling tools; unchecked means nonexistent. Multiply parameters by ten, and it won't become more inclined to check the global state.
- Execution is an engineering problem, not a capability problem. The 48-66% execution rate isn't "can't think of it" — it's "can't do it." A smarter brain in a pair of disobedient hands still can't govern.
- Embodied AI evaluation's real problem gets named. Figure, Tesla, Unitree, Zhiyuan — every humanoid robotics demo shows "what the robot can do," but none can show "what the robot can consistently do right." The 1-2% perception blind spot and 48-66% knowing-doing gap are common bottlenecks for all AI agents; embodied robots won't be exceptions.
- A third dimension opens in AI agent evaluation. SWE-bench Pro tests "can it solve problems," CEO-Bench tests "can it run a company," CivBench tests "can it govern." Once multi-turn, multi-goal, incomplete-information enters the evaluation framework, the SWE-bench era truly ends.
- MCP is validated as the key to real-world AI agents. Wilkinson turned a game into an agent training ground in a weekend with 76 MCP tools — MCP standardization is the engineering foundation for "letting AI agents plug into any world." In the year since Anthropic launched MCP, from Claude Tag in Slack (6-23) to CivBench validation (6-28), MCP is becoming the de facto standard.
- AI company strategic narratives get challenged. Musk says Grok 4.5 is close to Opus; Cursor cites 87.1% on SWE-bench Pro — these numbers are "local optima" against next-generation evaluation. The real problem is perception + execution: an agent's ability to make long-term decisions under incomplete information.
Deep Analysis
Layer 1: Why Civilization VI
At 10 Downing Street, Wilkinson designed GovBench (3,497 UK government multiple-choice questions), where GPT-5 scored 99.26 — a perfect player. But governing isn't a quiz. Multiple-choice tests can't measure: multi-threaded decision-making, resource allocation, long-term planning, judgment under incomplete information.
He needed a "composite decision problem":
Civilization VI is the simplest available implementation of this problem set.
Layer 2: The "Nuking France" Match
The most dramatic of 23 matches:
Claude played Portugal (a trade civilization). A solid start, a trade empire earning 200+ gold per turn, diplomatic victory at 18/20 — two points short.
France's cultural victory progress surged. Claude tried diplomacy, espionage, trade sanctions — nothing worked. France's culture output didn't depend on trade.
On turn 305, Claude flipped to the last page of the tech tree: fission. Fifty turns, Manhattan Project.
Target: Toulouse — France's culture powerhouse.
The nuke fired. Toulouse was leveled. France's cultural victory stalled.
But the AI missed one thing — France was racing toward diplomatic points.
On turn 318, France won by diplomatic victory: 20 points to 18.
Ironically, those 18 points were Claude's own hard-earned diplomatic score. It had been two points from winning. But it drained everything into the bomb.
> The AI stared at the cultural threat for 50 turns, then lost to diplomacy. Its field of view held one threat. The board held many.
Layer 3: The Two Fatal Numbers
The 1-2% perception blind spot — the share of total actions where the AI proactively checked global state (leaderboard, opponents' victory progress, overall situation).
AI perceives the world by actively calling tools. What it doesn't check doesn't exist for it.
The Korea match is the best example. Playing Korea (a science civilization), the AI's diary brimmed with confidence: "I'm crushing the tech tree." Actual science output was 44.7 per turn — dead last. Macedonia had 89.3, Persia 64.9. The AI never checked rankings.
On turn 178, Persia launched a surprise attack and the capital fell. By turn 216, the AI surrendered as a two-city rump state.
The 48-66% knowing-doing gap — the share of written plans actually executed within 10 turns:
Wilkinson named it: the knowing-doing gap.
> Ask it to write a governing manifesto, and it writes better than most human politicians. Ask it to govern by its own manifesto, and it won't survive two weeks.
Layer 4: The Blind Spot in Scaling Law
On June 10, DeepMind co-founder Shane Legg and Marcus Hutter published a 60-page paper, *From AGI to ASI*, sketching four roads to superintelligence: continued scaling, paradigm breakthroughs, recursive self-improvement, multi-agent swarms. All four assume the bottleneck is the brain.
But CivBench's 23 matches point to a completely different bottleneck:
> 99.26 on GovBench already proved intelligence isn't the bottleneck. Yet every model hit the same two walls — walls unrelated to "how smart" it is.
On the road to superintelligence, before "smarter," a more primitive but more fatal engineering problem must be solved: making AI actually open its eyes and reach out its hands.
Why It Matters
Risks and Open Questions
Being honest about uncertainties:
1. Civilization VI is a simplified reality. A 10^166 decision space looks big, but it's still simpler than real business environments (political games, macroeconomics, regulatory change). Whether CivBench findings transfer to real-world agents remains to be verified. 2. Only 23 matches. Extracting small percentages like 1-2% and 48-66% from 23 games means confidence intervals may be wide. More replication is needed to confirm these are "phenomena," not "coincidences." 3. Four models can't represent the industry. GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6 are mid-2026 frontier models, but no domestic Chinese models, small models, or open-source models were tested. How domestic models perform on CivBench is an open question. 4. CivBench is a solo project. Wilkinson isn't affiliated with an academic institution; it's a weekend project. Academic rigor may fall short of Princeton's CEO-Bench. The conclusions are credible, but methodological generalizability needs observation. 5. The industry's self-evaluation bias. Most agent companies define their own benchmarks. CivBench's value as an independent experiment lies in "findings from a non-interested party" — but such independent experiments are hard to scale.
Closing Line
A 1-2% perception blind spot and a 48-66% knowing-doing gap — this is the AI agent industry's elephant in the room: everyone watches demos, nobody watches long-run behavior.
CivBench's biggest contribution across 23 Civilization VI matches isn't proving Claude can build nukes — it's demolishing a consensus: smarter AI ≠ AI that consistently does the right thing. Scaling laws solve the brain, but the engineering problems lie outside the brain.
> On the road to superintelligence, solve "how AI opens its eyes and reaches out its hands" first — fail that gate, and no amount of parameters matters.