Anti-Self-Distillation: When AI Refuses to Copy Its Own Correct Answers, Reasoning Speed Jump 10x

Forum topic · 小凯 · 2026-05-24

Summary

Self-distillation is a common technique for improving the mathematical reasoning of large language models: when a model solves a problem correctly, its chain-of-thought is reused as training data. However, this approach often degrades reasoning. Researchers attribute the failure to "Teacher Confidence Inflation": when learning from solutions accompanied by correct answers, the model overfits to conclusive, connective filler text while losing deliberative tokens such as "wait", "let me think", and "maybe" that drive genuine problem exploration. AntiSD (Anti-Self-Distillation), published May 20, 2026 (arXiv:2605.11609), counters this by reversing the usual objective: instead of minimizing KL divergence to the answer-conditioned teacher, it deliberately maintains distance, uses pointwise mutual information (PMI) analysis to identify which tokens are filler versus decision-critical, and applies an entropy-force trigger that switches to a robust mode when teacher entropy collapses. Across five models from 4B to 30B parameters, AntiSD achieved 2x to 10x faster training, up to 11.5 points improvement on math benchmarks, and deeper deliberation, restoring independent reasoning in self-play RL without a stronger external teacher.

The "Smart Trap" of Self-Distillation

Teaching AI to do math often relies on a form of self-improvement: let the model solve problems, and when it gets them right, record its chain-of-thought (CoT) and train it on that process. This is called self-distillation. But the method has a strange failure mode: sometimes the model improves, sometimes it gets dumber. Why would referencing correct answers make reasoning worse?

It's like a student who only memorizes answers — change the question slightly in the exam, and their rigid logic falls apart.

The Root Cause: Spoiled by "Answers"

Digging deeper, researchers found the problem. When the model learns while looking at correct answers, it develops an illusion: it eagerly learns conclusive, connective filler text, while skipping the tentative words that actually drive problem-solving — phrases like "wait", "let me think", "maybe".

> 💡 This is called Teacher Confidence Inflation. The correct answer acts as a "spoiler": the model only learns to replicate the result, losing the "deliberating" quality of the derivation process.

The Fix: AntiSD's Contrarian Approach

Released May 20, 2026, AntiSD (Anti-Self-Distillation) takes a hard line: if correct answers distort the training signal, go the opposite way.

1. Reverse pull: instead of forcing the student to fit the teacher (the answer-conditioned model), maintain a subtle "centrifugal" distance. 2. Pointwise mutual information (PMI) analysis: precisely identify which tokens are filler and which are critical. 3. Entropy-force trigger: once the teacher becomes overconfident (entropy collapse), switch off the interference and enter a robust mode.

The core optimization logic is captured by this formula:

\nabla_\theta \mathcal{L}_{AntiSD} = - \eta \cdot \nabla_\theta \mathbb{D}_{KL}(\pi_{teacher} || \pi_{student})

> 💡 Note the negative sign ( $$-$$ ). Traditional learning minimizes the gap (descending); AntiSD deliberately widens it (ascending), letting the model recover its independent "spark" during self-training.

Results

Researchers ran AntiSD on five large models ranging from 4B to 30B parameters, and it won across the board. Logic that previously took thousands of training steps now emerges in a few hundred.

This means AI has finally found a way to "rescue itself". No stronger teacher watching over it is needed — with just the correct answers and this contrarian algorithm, the model can pull itself out of the logical swamp.

Reference

Paper: Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information
Published: May 20, 2026
Identifier: arXiv:2605.11609
Problem addressed: Loss of "Deliberation Tokens" and logical degradation caused by self-distillation in reasoning RL
Mechanism: The AntiSD strategy uses PMI analysis to identify and reinforce key decision points in the reasoning process