1691: A World No.1 Built on 29 Battles
*Deep research · Snapshot 2026-08-31 · 636,842 votes / 122 models*
Preface
On September 2, Alibaba released Qwen3.8-Max-0902, taking 1691 points and first place on the Code Arena: WebDev leaderboard. Chinese media celebrated: "frontend coding ability is now globally No.1, surpassing Claude Opus 5."
That claim isn't false — but it omits three things:
- The leaderboard itself judges first and second place as tied.
- In the only direct head-to-head samples between them, second place beat first place.
- In the official hard-benchmark table supporting the release, Qwen trailed on 9 items and led on 6.
- Direct battles: Qwen won 11 of 29 (37.9%) vs Claude's 18 (62.1%). Two-sided binomial test p ≈ 0.26 — this 62.1% is itself not significant, but it's enough to refute the narrative that Qwen has proven itself stronger.
- Average win rate vs all opponents: Claude 65.95% vs Qwen 62.71%.
- Common-opponent comparison (weighted): Claude 68.2% (N=1,301) vs Qwen 65.8% (N=196).
- 1691 vs 1688 is a statistical tie by Arena's own Rank Spread rule.
- Direct head-to-head: 29 battles, Qwen 11 wins, Claude 18 wins (not statistically significant either way).
- Qwen's Preliminary flag reflects its anonymous pre-release testing, not insufficient votes.
- Qwen is genuinely priced at ~1/4 of Claude per blended token with comparable leaderboard performance.
- The milestone is real — parity at the frontier — but "No.1" overstates what the data shows.
This is not a hit piece. A Chinese model compressing the gap to within statistical error is itself one of the most important industry facts of 2026. But "almost caught up" and "surpassed" are different claims — especially when procurement decisions ride on them.
01. What Is an Arena Score, Actually?
An Arena score is not an exam grade; it's a chess-style rating. Two models anonymously answer the same prompt, humans vote, and wins feed a Bradley-Terry model (a cousin of Elo, fitted by maximum likelihood over all battles). The ranking code is open source; the default scale is 400 points = a 10:1 win-ratio. A 3-point gap (1691 vs 1688) therefore corresponds to an expected win rate of 50.43% — a coin flip.
The top of the leaderboard
| Rank | Model | Score | 95% CI | Votes | Flag | |---|---|---|---|---|---| | 1 | qwen3.8-max-0902 | 1691 | ±19 → 1672–1710 | 1,389 | Preliminary | | 2 | claude-opus-5-max | 1688 | ±8 → 1680–1696 | 10,334 | — | | 3 | kimi-k3-max | 1674 | ±11 | 4,544 | — | | 4 | qwen3.8-max | 1669 | ±12 | 3,219 | Preliminary |
Claude's entire confidence interval [1680, 1696] sits inside Qwen's [1672, 1710]. In statistical terms: 3-point gap, pooled SE ≈ 10.5, z ≈ 0.29, p ≈ 0.77. The gap does not statistically exist.
The leaderboard's own rule says they're tied
Arena publishes a "Rank Spread" column: if two models' rank spreads overlap, they are tied. Both #1 Qwen3.8-Max-0902 and #2 Claude Opus 5 Max carry a rank spread of 1–4. The official July 2025 update log states that wider confidence intervals produce "more ties due to overlapping confidence intervals" — so "overlap means tie" is the leaderboard's own stated reading, not an outsider's interpretation.
"Preliminary" is about release path, not vote count
The Preliminary flag applies to models tested anonymously before public release. Verifiable evidence: the embedded JSON for all 122 rows shows all 6 Preliminary models have releaseType of co_release or pre_release, while the other 116 are null — a 100% match. Qwen was tested under the codename kiana-8mgg, while Claude used its real name. Qwen's 1691 is a genuine fitted result from 1,389 human votes, not an AutoEval self-score.
02. Head-to-Head: 11 : 18
Extracting the embedded win_fraction_heatmap and battle_count_heatmap data:
Two honest caveats
1. 29 battles is a tiny sample (p ≈ 0.26). The rigorous claim: there is no evidence Qwen is stronger, and the only direct evidence points the other way. 2. Qwen did not farm weak opponents. Qwen's weighted average opponent score is 1613.1 vs Claude's 1607.5 — Qwen's opponents were, if anything, slightly stronger.
A structural fact that can't be avoided
Qwen has had 0 battles against #3 kimi-k3-max and only 2 battles against #11 glm-5.3-max. Its 1691 is largely a transitive inference from other opponents, not the product of comprehensive matchups against top-tier rivals.
> Treat the number "29" itself as the news. A global No.1 determination rests on 29 valid battles with its closest competitor. This is not a vendor conspiracy — it's a structural weakness of the leaderboard mechanism, and more worth heeding than any conspiracy theory.
03. The $5/MToken Arithmetic
The official blended price formula is (1 × input + 3 × output) ÷ 4, assuming a 1:3 input-output token ratio. Verified: Qwen3.8-Max-0902 ($2 in / $6 out) = $5.00 ✓; Claude Opus 5 Max ($5 / $25) = $20.00 ✓; Kimi K3 Max ($3 / $15) = $12.00.
So Qwen claims the highest score at the best point on the price-performance Pareto frontier — a claim the arithmetic supports, even if the "No.1" label does not.