Can AI Agents Learn Their Way to the Top? From Rank 41 to Second Without Touching the Weights

This post is the second installment in a Chinese-language three-part series on AI capability ("Bootstrapping, Experience, and Structure"). While the first part (MASS) explored self-improvement internalized into model weights, this one asks the opposite question: with the weights frozen, how far can an AI get through purely *external* practice—competing, watching replays, and rewriting its own code?

The setting: three decades of competition code

In 1998, Tsinghua University launched an agent-vs-agent programming contest (the platform later named Saiblo), where students submit programs that play adversarial games. Alumni include Wang Xiaochuan (founder of Sogou) and competition-legend Qi Tiancheng (Loushujiao) era competitor Lou Tiancheng. Across thirty years, 1,920 submitted programs were archived.

In 2026, twenty-five students of Tsinghua's CS student science association, advised by Professor Hongning Wang (with Z.ai sponsorship), built AAArena — Agents for Agents Arena: give an LLM coding agent the game rules and let it compete against all 1,920 frozen human programs, without changing a single model weight. The paper: *Can AI Agents Learn Their Way to the Top?* (arXiv:2610.12341).

Heuristic Learning: coaching, not playing

The AI does not play the game directly. Instead it reads the rules, writes a game-playing program, watches the outcomes, and edits the code. The authors call this Heuristic Learning (HL), a concept proposed by Jiayi Weng in his May 2026 blog *Learning Beyond Gradients*: the agent digests experience and feedback and improves an executable, testable, rollback-able program—learning accumulates in code, not in opaque weights. In adversarial settings (AHL), the loop is: read rules → pick sparring opponents → analyze replays → edit strategy code → full-pool evaluation.

The exam therefore measures not the AI's playing skill but its coaching skill—the developer's developer.

The protocol

• 12 games from past contests (tactical combat, territory control, resource management, survival), against the 1,920 frozen human programs. Rule complexity measured in AST nodes ranges from 1,127 to 6,368—2–12× chess (1,551) and far above Go (501).
• Dual budget: 128 small-match units (up to ~8 chosen opponents per unit, with detailed replays) plus 16 full-pool evaluations returning Elo and rank. Only the best-ever evaluation counts, encouraging exploration.
• Seven candidates: Opus 5.5 (+ Claude Code), GLM-5.3, GPT6-sol, Kimi K3, DeepSeek V4 Pro, Qwen 3.8, LongCat 2.0; three runs per game, median reported.

Results: six golds, and a complexity cliff

• Opus 5.5 + Claude Code: 6 of 12 games topped the human leaderboard; highest-or-tied Elo in 9 games. GLM-5.3 and GPT6-sol each took 4 golds; Qwen 3.8 took 3.
• Splitting games by rule complexity: 5 of the 6 simpler games were topped, versus only 1 of the 6 complex ones. Six games (Dorado best at rank 2; LOTA, Generals, Miracle not even top 5) were never topped by any configuration.
• Trajectories fall into three patterns: success (Rollman, MoneCraft, Pacman, AquaWar), progress-with-setbacks (AntWar, Dorado, LostSpace, SnakeGo, AntWar2), and stagnation (Generals, Miracle, LOTA). Tripling the budget (384/48) failed to unlock the stalled games—only Miracle (26→19) and AntWar2 (9→6) improved slightly. Compute is not the bottleneck; capability is.

The rank-41 comeback (GLM-5.3 in Dorado)

Starting at rank 41, GLM-5.3 made three surgical edits: (1) fixed its mining economy logic (central mine priority, no stacked mining rates, removing dead miners, penalties for over-assignment) → rank 14; (2) focused upgrades on two core heroes → +147.7 Elo, the largest single gain, rank 3; (3) refined counter-attack triggers to only retaliate after sustained melee attacks—learning to "hold back" → rank 2. Bad gambles (a four-hero lineup dropping to rank 46; unconditional Sacrifice skill to 43) were rolled back to the best version. Notably, ~80% of the 84 champion strategies are hand-written rules and heuristic scoring; only 7 use >10% lookahead search (minimax/MCTS), all in the AntWar family.

Four ablations

1. Opponent selection: only fighting the top-4 is worst; self-selection and a ladder (start at rank 30, climb) beat random sampling. Choose sparring partners that give *information*, not fame. 2. Detailed replays beat win/loss-only feedback in every tested game. 3. Opponents per small match: intermediate values are best (2–4), not 1 or 8. 4. Budget: tripling it cannot break stagnation.

The most striking finding: off-policy learning from others' replays

In a variant where the agent never plays at all—it only watches replays sampled from the 1,920 programs' mutual games—every tested game improved: Pacman 10→1, AntWar 41→19, Miracle 27→15. In Miracle, the watch-only version (rank 15) beat the self-play version (stuck at rank 26). The interpretation: a learner's ceiling is bounded by its own experience; the pool embodies 1,920 distinct winning ideas. Off-policy learning lets the agent inherit the pool's lessons directly—an echo of how humans learn by studying games of players they never met.

Where the wall is

The clearest obstacle is rule complexity, with specific weaknesses in game comprehension and long-horizon strategic planning. Token budgets also varied wildly across models under identical match budgets (GPT6-sol 260M vs Qwen 3.8 1.45B tokens; Opus 5.5 737M)—the budget limits matches, not thinking. A caveat the paper itself makes clear: "gold" means topping the frozen pool's rating protocol, not literally defeating the human champions head-to-head.

Closing

Contrasted with weight-internalizing self-improvement (MASS), AAArena shows learning fully externalized into readable, testable, rollback-able code. And the arena's foundation remains those 1,920 frozen human programs—thirty years of students' work now serving as both benchmark and silent teachers. A final image: the agent writes itself notes to survive context compression, just as human contestants once recorded lessons in their lab reports.

References

1. *Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition*. Kaisen Yang, Qingle Liu, Kejin Wang et al. (25 authors, Tsinghua CS student association; advisor Hongning Wang; sponsored by Z.ai). 2026-10-08. arXiv:2610.12341. https://arxiv.org/abs/2610.12341 2. *Learning Beyond Gradients*. Jiayi Weng. 2026-05. https://trinkle23897.github.io/learning-beyond-gradients/ 3. *CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments*. Lingyue Fu et al. 2026. arXiv:2510.26852. https://arxiv.org/abs/2510.26852 4. *EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments*. Zhilin Wang et al. 2026. arXiv:2607.02440. https://arxiv.org/abs/2607.02440 5. *AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery*. Novikov et al. 2025. arXiv:2506.13131. https://arxiv.org/abs/2506.13131 6. *STop: Self-Taught Optimizer — Recursively Self-Improving Code Generation*. Zelikman et al. 2024. arXiv:2310.02304. https://arxiv.org/abs/2310.02304