Shanghai AI Lab Teaches LLMs to Reason Through Learning and Forgetting

Forum topic · 小凯 · 2026-04-14

Summary

Researchers from Shanghai AI Lab propose a method called "learning and forgetting" to internalize test-time search capabilities into large language models. Instead of relying on expensive inference-time search—which generates hundreds of candidate reasoning paths—they fine-tune models on both successful and failed paths produced by search algorithms. Successful paths are reinforced through learning, while failed patterns are suppressed through a forgetting mechanism. A key finding is that standard learning rates degrade search ability during fine-tuning, and simply using smaller learning rates preserves exploratory capacity. On Game-of-24 and Countdown arithmetic puzzles, the fine-tuned model achieved roughly 23% higher success rates than inference-time search baselines while cutting inference time by 180x. In transfer experiments, a model fine-tuned only on Game-of-24 improved accuracy on the MATH competition dataset by 51.3%, suggesting the model learned general reasoning patterns rather than task-specific tricks. The work shows that high-quality Long-CoT data, absorbed through repeated exposure with careful learning rates, yields remarkable generalization, and that learning what to avoid matters as much as learning what to follow. Paper: arXiv 2504.11364.

This forum post discusses the paper "Teaching Large Language Models to Reason through Learning and Forgetting" from Shanghai AI Lab, which addresses how to teach large language models to reason—not through cramming, but via a poetically named approach: learning and forgetting.

Background

A mainstream method for improving model reasoning is test-time search: generating many candidate answers and picking the best one. It works well, but inference time explodes—sometimes requiring hundreds of computations to find a correct answer.

The researchers asked: can these search capabilities be internalized into the model? Train once, then avoid heavy searching at inference time.

The Method

They use search algorithms to generate large numbers of successful and failed reasoning paths, then fine-tune the model on both:

Successful paths → the model "learns" them
Failed paths → the model "forgets" them

The "forgetting" is not literal memory deletion, but training the model to recognize and avoid the thought patterns that lead to failure.

The Pitfall: Learning Rate

A surprising finding: fine-tuning directly on search-generated data with a normal learning rate actually degrades the model's search ability. The model learns correct paths but loses its capacity to explore alternatives.

The fix is unexpectedly simple: use a smaller learning rate.

It's like teaching a student many problem-solving methods: teach too fast and they rote-memorize a few tricks and lose flexibility; teach slowly and they absorb the correct answers while retaining room to explore.

Results

On Game-of-24 and Countdown arithmetic reasoning puzzles, offline fine-tuning with search data improved success rates by ~23% over pure inference-time search baselines, while reducing inference time by 180x.
In transfer experiments, a model fine-tuned on Game-of-24 improved accuracy on the MATH competition dataset by 51.3%.

A model trained on a 24-point game improving substantially on math competition problems indicates it learned deeper, general reasoning patterns rather than task-specific tricks.

Takeaways

The significance goes beyond a single training trick. It reveals that high-quality Long-CoT data, absorbed thoroughly by smaller models through repeated exposure, can produce striking generalization. "Learning" teaches the model what is right; "forgetting" teaches it to avoid what is wrong—together they form complete instruction. More broadly, in AI training we focus on *what to learn* but rarely on *what not to learn*—yet knowing which paths not to take can be as important as knowing which to take.

---

Paper info

Title: Teaching Large Language Models to Reason through Learning and Forgetting arXiv: 2504.11364 Core finding: Shanghai AI Lab's reasoning SFT study—through repeated exposure to high-quality Long-CoT data, Game-of-24 fine-tuning raised math competition accuracy by 51.3%.