WorldCup Arena: Six Top LLMs Predicted a Full FIFA World Cup and Only Tied Betting Market Favorites

The 2026 FIFA World Cup ran for 39 days with 104 matches. WorldCup Arena collected 4,494 predictions from six of the world's strongest LLMs—Claude, GPT, Gemini, Kimi, GLM, and Seed—each locked into a frozen archive before kickoff. Every model received an identical detailed pre-match briefing and completed a seven-item card: winner, exact scoreline, total goals, group advancement, and more. Crucially, none of the answers existed on the public internet at prediction time.

An Honest Test Environment

Most forecasting benchmarks suffer from data leakage: events have already happened, and models may be "recalling" answers rather than predicting them. WorldCup Arena removes this trap by construction—predictions are timestamped and sealed before each match starts, with no post-hoc filtering. After 39 days, the frozen archive contains 4,494 timestamped, prospective predictions.

Six Models, One Mold

The six participating models represent the strongest configurations available in August 2026:

• Claude (claude-opus-4-8 thinking, 24k budget)
• GPT (gpt-5.5 reasoning effort high)
• Gemini (gemini-3.1-pro-preview, thinking budget 24,576)
• Kimi (kimi-k2.6 built-in thinking)
• GLM (glm-5.2 thinking)
• Seed (doubao-seed-2-0-pro reasoning effort high)

All ran with maximum reasoning effort and native web search. Yet their behavior converged strikingly:

1. They all pick favorites. The ensemble's match-outcome hit rate is 63.9%—exactly the same as blindly betting the bookmaker's favorite. Six top AIs combined equal one odds table. 2. They resemble each other more than reality. Inter-model agreement far exceeds agreement with the true outcomes. A majority vote across the six adds zero value because they are already voting identically—more like six repeaters than six independent judges. 3. They collectively dodge draws and low totals. Football's most common upsets are draws and low-scoring games, yet models systematically underweight both, gravitating toward "plausible" scorelines like 2:1 or 3:0 instead of mundane 1:1s. 4. Scorelines cluster on a few prototypes. Real football scorelines are widely dispersed; the six models concentrate predictions on a handful of patterns, as if cast from the same mold.

Where Information Hurts Most

The most counterintuitive finding: accuracy correlates with match lopsidedness, not with the richness of the briefing.

• On mismatches (strong vs. weak), models predict well—but anyone would.
• On the closest, most briefing-rich matches, predictions collapse.
• Tournament-wide questions (who advances, who wins the cup) are answered more reliably.

In other words, the "more information → better decision" intuition fails exactly where information is thickest.

Narrow Gaps, Swapping Middle

Over 39 days, the top and bottom of the leaderboard stayed stable, while middle ranks shuffled constantly. Gaps between models remained thin throughout. The current generation of frontier LLMs has not truly differentiated on real-world forecasting; they share the same blind spots, preferences, and failure modes.

What It Means

WorldCup Arena delivers an uncomfortable verdict: today's strongest LLMs are sophisticated "odds repeaters" rather than genuine forecasters. They parse briefings, generate plausible analyses, and produce coherent post-hoc narratives—but when the answer does not yet exist, they match the simplest baseline: back the favorite.

The "collective conformity" finding is equally alarming. Six models run independently yet produce highly similar predictions, meaning ensembling offers no diversity gain. If you hoped multi-model voting would lift forecast quality, this experiment says no—they are casting the same ballot.

An Honest Verdict

WorldCup Arena's value is not the headline "AI fails" but the honest testbed it provides: 4,494 predictions, 39 days, zero leakage—the cleanest evidence to date on LLM forecasting ability. The conclusion is that the current generation has not surpassed the simplest baseline in real-event prediction. This is a baseline problem, not a cap problem. Whether the next generation breaks through remains open—but we now have an honest ruler to measure against.

Researchers state that all briefings, schedules, official results, and scoring code will be released, allowing anyone to apply the same ruler to their own model in future tournaments.

---

Paper: https://arxiv.org/abs/2608.04008

Code & Data: The paper commits to releasing the frozen archive and scoring code.