SU-01 Deep Dive: How a 30B-Parameter Model Wins Olympiad Gold with a Simple Unified Recipe

This post analyzes the paper "Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling" (arXiv:2605.13301), from Shanghai AI Lab together with CUHK, Tsinghua, SJTU, and Peking University. The core claim: Olympiad-gold reasoning does not require a bigger model — it requires a better training recipe.

Headline results

• IMO 2025: 35/42★ (★ = human gold-medalist review), exactly the gold-medal line; direct generation only scored 21
• USAMO 2026: 35/42★, 10 points above the gold line, matching the top human score among 340 contestants (median: 6)
• IPhO 2024/2025: above the gold line after test-time scaling (TTS)
• IMO-ProofBench: 57.6% direct (ties Gemini-2.5-DeepThink), 70.2% with TTS
• Model: 30B-A3B MoE (3B activated parameters), built on the P1-30B-A3B base
• Cost: 338K SFT trajectories + only 200 RL steps

Key points

• Specializable Generalist philosophy: instead of training a narrow specialist from scratch, SU-01 lightly adapts a generalist science base (P1-30B-A3B), reshaping its behavior from "answer quickly" to "rigorous proof search" without catastrophic forgetting of general abilities.
• Data quality over quantity: 338K trajectories — Math 71.8K (21.2%), STEM 62.9K, Code 30.2K, IF 18.8K, plus Self-Verify 89.5K + Self-Refine 65.2K = 45.7% behavior data that teaches the model *how to check itself*, not just answers. Data is hard-truncated to sub-8K tokens for training stability.
• Inverse-perplexity curriculum (the first key innovation): ordering SFT samples by *descending* perplexity (hardest first) reached 55.8 on AnswerBench / 40.0 on AMO-Bench versus 24.3 / 15.0 for ascending order and 39.5 / 31.0 for random. Training the model on unfamiliar reasoning patterns first, then consolidating with easy samples, yields clean adaptation (validation truncation rate 0–0.3% vs. a <5% target). 4 epochs, LR 1e-5 → 1e-6 cosine, weight decay 0.1, Adam β2=0.95, batch 128.
• Two-stage RL:
• *Coarse RL* (96 steps, 8,967 verifiable prompts, GSPO with sequence-level importance ratios, K=8 samples, 160K max length): a three-layer reward — rule-based text matching → Math-Verify → generative verification via gpt-oss-120b (only as conservative fallback, saving API cost).
• *Refined RL* (104 steps, 25,254 prompts): binary generative proof reward from DeepSeekMath-V2 on mathematical validity, rigor, and completeness. When the group mean proof reward < 0.5, 20% of queries get a single, non-recursive correction pass.
• *Anti-hack*: detects leaked chat-template tokens, unbalanced thinking delimiters, heavy repetition.
• *Experience replay* (25%): retains problems with exactly one success (hard-but-learnable), retires those solved ≥4 times, and replays the lowest-entropy successful trajectory to avoid lucky-guess contamination.
• Test-time scaling: a Solve → Verify → Refine → Adjudicate loop with stop rules (5 consecutive verification-true rounds to accept, 10 false rounds to abort, up to 30 exploration rounds and 10 independent runs). On USAMO 2026, median lengths were 106K tokens (initial solve), 83K (refine), 28.7K (verify), 404 (adjudicate) — a problem can total 200K+ tokens.
• Train–inference length asymmetry is intentional: SFT at 8K teaches the *behavior* (search, verify, refine); inference at 100K+ unfolds deeper search within that learned framework — like learning chess strategy on short games, then playing a full game.

Why it matters

SU-01's recipe is "minimal" in three ways: one unified pipeline for math and physics (no domain-specific systems), 338K trajectories instead of millions, and only 200 RL steps (RL amplifies behaviors already shaped by SFT rather than learning from scratch). Trained only on math/physics signals, it still generalizes to chemistry and biology on FrontierScience-Research (11.7% overall — best in its size class, though far behind GPT-5.5-High's 36.7%).

Comparison snapshot:

| Dimension | DeepSeek-R1 | SU-01 | |---|---|---| | Scale | 671B | 30B-A3B | | Data | millions of RL trajectories | 338K SFT + 200 RL steps | | Verification | rule-based rewards | layered rule→LLM + generative proof reward | | TTS | minimal | Solve-Verify-Refine loop |

Limitations

• P6 = 0 on both IMO and USAMO: TTS lifts "solvable but error-prone" problems but cannot break the intrinsic difficulty ceiling of the hardest problems.
• IPhO scores lack human expert review; scoring details (units, significant figures, partial credit) may overstate performance.
• FrontierScience absolute scores remain low; chemistry/biology ability comes from base-model generalization, not targeted training.
• TTS inference cost is nontrivial: the model is small, but 100K+ token generations can approach the cost of short runs on much larger commercial models.

Takeaway

Scaling laws have more than one axis. Beyond model size and data volume, method scaling — curriculum design (inverse perplexity), staged RL, and test-time verification loops — can let a 30B model match systems an order of magnitude larger. When resources are limited, smarter training may be the better investment than scale.

Reference: Li, Y., Zhan, R., Zhang, H., et al. (2026). *Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling*. https://arxiv.org/abs/2605.13301