Self-Verified Distillation: Stanford Paper Shows LLMs Can Grade Their Own Homework and Improve

The Paper

| Item | Detail | |------|--------| | Title | Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline | | Authors | Tony Lee, Percy Liang | | Institution | Stanford University | | arXiv ID | 2605.26132 | | Submitted | May 20, 2026 | | Category | cs.CL |

The author opens with an analogy: a student who solves math problems with no answer key, then copies their solution three times and checks each line for skipped steps, unjustified assumptions, and sign errors—keeping only the versions that pass self-review. The paper makes language models do exactly this, and it works.

The Core Idea

Starting from only unlabeled questions, the model generates n candidate solutions per problem. Each candidate must pass three verification gates, all of which are oracle-free (no ground-truth answers needed):

1. Cycle-consistency: hide the solution, give the model only the final answer, and ask it to reconstruct the problem conditions. If the reconstruction matches the original problem, the solution is at least internally coherent. This filters out incoherent generations. 2. Factuality: every factual claim (theorems, formulas cited) is independently checked v times; only unanimous agreement passes. 3. Correctness: multi-angle logical review of the solution—hidden assumptions, boundary conditions—again with unanimous vote across repeated judgments.

For hard seed problems, out of 100 candidates only a handful (or none) may survive all three gates.

The verification design builds on the Unsolved Questions (UQ) benchmark (Allen Institute for AI, 2025), repurposing its oracle-free composite validators from an evaluation tool into a training filter. The author notes an implicit assumption: filters useful for human evaluation may not transfer to models, since model errors can be systematic and correlated with the validator's own tolerances.

Key Findings

• Unfiltered self-training can hurt. Training on all self-generated solutions without verification *degraded* math, science, and coding performance—a rare honestly-reported negative result in the self-improvement literature (STaR, Quiet-STaR, etc.).
• With strong filtering, gains are large. Qwen3-4B pass@1 improved by +16.7 on math (AIME26, HMMT), +11.1 on science (GPQA Diamond, HLE), and +8.3 on coding (LiveCodeBench v5/v6). Qwen3-0.6B and 8B also improved, though 0.6B results were noisier—suggesting a capability threshold.
• Training-time compute amortizes test-time compute. The method matches or beats UQ-TTC (test-time-compute baseline using the same three-stage verification) while requiring only a single inference call at test time. The author frames this as knowledge compression: expensive multi-path verification is baked into parameters.

Data and Cost

• Seed questions come from OpenThoughts: 53,125 math, 26,041 science, 9,168 coding problems—with reasoning traces and answers deliberately discarded to preserve the unlabeled setting.
• Verification cost is substantial: with n=100 candidates and v=3 judgments per gate, one math problem alone could require ~53,125 × 100 × 9 ≈ 47.8 million LLM calls at scale. This pays off only if the trained model serves many queries.
• Qwen3 was chosen (0.6B/4B/8B), likely for strong math/coding baselines, open weights, and long-context support (32,768 max output tokens).

Open Questions Raised by the Author

• Fragile unanimous vote? All three validators share the same parameters; correlated errors could let false positives through unconditionally. The paper does not quantify validator false-positive/negative rates.
• Seed quality boundary: OpenThoughts is a curated, reasoning-oriented dataset. No ablation on noisier seeds.
• Scale threshold: 0.6B struggles; where exactly does the capability floor sit?
• Relation to RLVR: complementary (generate data, then RL fine-tune) or a replacement for labeled-reward RL? Not discussed in the paper.
• Sampling temperature: generation and verification both use temperature 0.8, top-p 0.95. Correlated sampling errors across repeated judgments could undermine unanimous voting.
• Coding lags: +8.3 vs +16.7 in math—possibly because code already has deterministic verifiers (compilers, unit tests), making LLM-based validation less valuable there. The method's biggest value may lie in domains *without* deterministic verifiers.

Alternative Paths Discussed

The author explores counterfactual designs: weakening verification in exchange for data volume (majority vote instead of unanimous); external verifiers or tool execution; continuous quality weighting instead of binary filtering; and iterative generate-critique-revise loops. Lee and Liang's chosen path trades data efficiency and engineering simplicity for conceptual purity: a closed system that extracts its own training signal from questions alone.

Closing Metaphor

The author reframes "distillation" not as knowledge *transfer* (Hinton, 2015) but as knowledge purification—with the twist that the purity standard is defined by the model itself and co-evolves as the model improves, closer to Socratic self-questioning than chemical distillation.