HEAVYSKILL: Turning Heavy Thinking into an Internalized Self-Debate Skill for LLMs

Forum topic · 小凯 · 2026-05-23

Summary

HEAVYSKILL (paper: 'Heavy Thinking as the Inner Skill in Agentic Harness', arXiv:2605.02396) reframes deep reasoning as a learnable skill rather than an external orchestration trick. It replaces Best-of-N majority voting with a two-stage pipeline: parallel reasoning, where K isolated thinkers (temperature 1.0) solve a problem independently, and sequential deliberation, where a deliberator critically reviews all trajectories via a structured, shuffled memory cache and can re-derive the answer when all thinkers fail. On AIME25, BeyondAIME, HMMT25-Feb, and GPQA-Diamond, it consistently beats majority voting—e.g., GPT-5 Thinking +9.5% on BeyondAIME, DeepSeek R1-0528 +6.7% on AIME25, with several 100% results. Gains are smaller on code tasks with strong verification signals but large with tool use (Qwen3-8B +15.2% on HMMT25). Ablations show trajectory quality/diversity matters, deliberation rewards general analytical capability over raw reasoning, and 2–3 deliberation iterations are optimal. RLVR training (VeRL + GSPO) improves K=8 by ~10% but suffers entropy collapse at K=16 due to context-length limits.

Paper: Heavy Thinking as the Inner Skill in Agentic Harness Authors: Jianing Wang, Linzin Guo, Zhengyu Chen et al. arXiv: 2605.02396 | Code: github.com/wjn1996/HeavySkill

Key points

HEAVYSKILL turns "heavy thinking" from an external orchestration byproduct into a learnable skill internalized in model parameters, replacing majority voting (Best-of-N) with critical deliberation over reasoning *processes*, not just answer counts.

Two-stage pipeline

1. Parallel reasoning: K independent thinkers (K = 8 or 16 in main experiments; 3–5 in agentic harnesses) solve the problem in isolation at temperature 1.0, top-p 0.95, encouraging diverse strategies. Isolation prevents conformity contamination. 2. Sequential deliberation: a deliberator receives a serialized memory cache of all trajectories (pruned for length, shuffled to avoid position bias) and must:

classify the query type,
critically evaluate every thinker without blindly following the majority,
re-derive the answer from scratch if all thinkers are wrong,
keep output format consistent (\boxed{} for math, code blocks for code).

Superficial copy-paste of majority answers is explicitly forbidden.

Results: consistently beats Best-of-N

Performance hierarchy found across 10+ models: Heavy-Pass@k ≥ Heavy-Mean@K ≥ Vote@K ≥ Mean@k.

| Model | Dataset | Vote@K | HM@4 | Gain | |-------|---------|--------|------|------| | GPT-5 Thinking | BeyondAIME (K=16) | 73.0% | 82.5% | +9.5% | | GPT-5 Thinking | HMMT25-Feb (K=16) | 86.7% | 95.0% | +8.3% | | DeepSeek R1-0528 | AIME25 (K=16) | 90% | 96.7% | +6.7% | | DeepSeek R1-0528 | HMMT25-Feb (K=16) | 83.3% | 91.7% | +8.4% | | Kimi K2 Thinking | GPQA-Diamond (K=16) | 80.3% | 87.5% | +7.2% | | GLM 4.6 | HMMT25-Feb (K=8) | 96.7% | 100% | +3.3% |

Several 100% results: Kimi K2 Thinking on AIME25 (K=8), DeepSeek V3.2 Thinking on AIME25/HMMT25-Feb (K=16).
GPQA-Diamond gains are small (+0.8–2.0%) due to ceiling effects; LiveCodeBench gains are modest (+0.3–2.5%) because code has strong verification signals.
With tool use, gains are large: Qwen3-8B +15.2%, GPT-OSS-20B +12.4% on HMMT25 (Vote@4 vs HM@4).

Ablation findings

Trajectory quality and diversity are key, but explicit diversity optimization adds no marginal benefit; random sampling suffices. Longer trajectories are *not* better (Max-Length selection is worst; Max-Answer-Num is best at K=64).
Deliberator capability matters: Qwen2.5-32B-Instruct (weak solo reasoner, 12.8% on AIME25) outperforms R1-Distill-Qwen-7B as deliberator (60.0% vs 50.0%, K=8), suggesting deliberation relies on analytical/synthesis ability—thinking and deliberation models can be separately optimized.
Iterative deliberation improves HM@K for 2–3 rounds but degrades HP@K as later steps get biased by earlier judgments.

RLVR training

Setup: VeRL + GSPO on R1-Distill-Qwen-7B, training on pass rate ∈ [0, 0.625] queries.
K=8: ~10% HM@4 gain, stable training. K=16: severe entropy collapse after ~100 steps due to the 7B model's context-length limit—longer-context models are needed.

Readable Skill for agentic harnesses

HEAVYSKILL ships as a plain-text skill (activation conditions, parallel protocol, deliberation prompt, output constraints) injectable into Claude Code / Codex-style harnesses. Crucially, deliberation must be executed by the main agent, not delegated to subagents, since it requires full context.

Limitations

Context length bottlenecks K=16 RL training.
Cost: K+1× token consumption vs single-pass inference.
Diminishing returns where strong external verification exists.
Iteration interference beyond 2–3 rounds; small models may see no gain (HM@4 below Vote@K for R1-Distill-Qwen-7B on some tasks).

Takeaway

HEAVYSKILL upgrades democratic voting into expert adjudication: parallel divergence plus critical synthesis. Its philosophical claim: heavy thinking should not only be an execution unit in orchestration harnesses but an inner skill internalized in model parameters via RL—external frameworks activate the skill; parameters execute it.

References:

Paper: arxiv.org/abs/2605.02396
GitHub: github.com/wjn1996/HeavySkill
Related: Hogwild! Inference, ParaThinker, Pacore; RLVR frameworks VeRL, GSPO