The paper *Teaching Large Language Models to Reason through Learning and Forgetting*, from Shanghai AI Laboratory, addresses how to teach large language models to "reason" — but not through rote drilling. Their method has a poetic name: Learning and Forgetting.
Background
A mainstream approach to improving model reasoning is inference-time search: generate many candidate solutions, then pick the best one. This works well, but inference time explodes — sometimes hundreds of passes are needed to find a correct answer.
The researchers asked: can this search capability be internalized into the model? Train once, and afterwards the model no longer needs extensive search.
The Approach
Use search algorithms to generate large numbers of successful and failed reasoning paths, then fine-tune the model on both:
- Successful paths → the model "learns" them
- Failed paths → the model "forgets" them
- On Game-of-24 and Countdown arithmetic puzzles, offline fine-tuning on search data improved success rates by ~23% over pure inference-time search baselines, while cutting inference time by 180x.
- In transfer experiments, a model fine-tuned on Game-of-24 alone improved accuracy on the MATH competition dataset by 51.3%.
- Title: Teaching Large Language Models to Reason through Learning and Forgetting
- arXiv: 2504.11364
- Core finding: Shanghai AI Lab's reasoning SFT research; through repeated exposure to high-quality Long-CoT data, fine-tuning on Game-of-24 improved math competition accuracy by 51.3%
The "forgetting" is not literal memory deletion; rather, the model learns to recognize and avoid the thought patterns that lead to failure.
The Pitfall: Learning Rate Matters
In practice, the researchers found a major trap: fine-tuning directly on search-generated data with a normal learning rate actually degrades the model's search ability. The model learns some correct paths but loses the capacity to explore alternatives.
The fix is surprisingly simple: use a smaller learning rate.
It's like teaching a student many solution methods. Taught too fast, they memorize a few routines and lose flexibility. Taught slowly, they absorb the material — mastering correct answers while keeping room to explore new approaches.
Results
Experiments are striking:
A model that learned a card game improved substantially at math competition problems — evidence it acquired lower-level reasoning patterns, not puzzle-specific tricks.
Significance
This is more than a training trick. It reveals a deeper principle: high-quality Long-CoT data, when absorbed by smaller models through repeated exposure, can produce remarkable generalization.
"Learning" teaches the model what is right; "forgetting" teaches it to avoid what is wrong. Together, they form complete teaching.
The broader lesson: in AI training we focus heavily on *what to learn*, but rarely on *what not to learn*. Sometimes knowing which paths not to take is just as important as knowing which ones to take.
---
Paper info