LIMR: How 1,389 Carefully Selected Math Problems Beat 8,523 in RL Training

Forum topic · 小凯 · 2026-05-11

Summary

In February 2025, researchers from the GAIR Lab at Shanghai Jiao Tong University published LIMR (arXiv:2502.11886), demonstrating that reinforcement learning (RL) fine-tuning of Qwen2.5-Math-7B with just 1,389 selected problems outperforms training on the full 8,523-problem MATH dataset. Their method, Learning Impact Measurement (LIM), scores each training sample by how closely its reward trajectory aligns with the model's average reward curve, filtering out samples with no teaching value (threshold θ=0.6). The resulting 16.3% subset achieved 78.0% on MATH500 versus 76.6% for full data, 63.8% vs 61.9% on AMC23, and parity at 32.5% on AIME24, while a randomly sampled subset of equal size lost 8.1% average performance. LIMR also substantially outperformed SFT-based approaches like LIMO and s1 on small models, suggesting data quality—not quantity—drives RL scaling, and that much of the industry's large-scale RL data collection may be wasted effort.

1. A Counter-Intuitive Experimental Result 🤯

In February 2025, researchers at SJTU's GAIR Lab did something that made many AI engineers want to throw their keyboards.

They took a base math model, Qwen2.5-Math-7B, and trained it with reinforcement learning to solve competition math problems. They used two versions of the training dataset:

| Dataset | # Problems | Share | |:------:|:-------:|:----:| | MATH-FULL | 8,523 | 100% | | LIMR | 1,389 | 16.3% |

Then they had both models take the same exam.

The result? The model trained on 16% of the data scored higher on average.

Not roughly equal—better. On MATH500, LIMR scored 78.0% vs. 76.6% for the full dataset. On AMC23, LIMR got 63.8% vs. 61.9%. Only on the hardest benchmark, AIME24, did the two tie (32.5%).

> MATH500: A curated set of 500 competition-level math problems from the MATH dataset, a standard benchmark for mathematical reasoning. > > AMC23: American Mathematics Competitions 2023, a national contest for high school students. > > AIME24: American Invitational Mathematics Examination 2024, harder, only 30 problems.

This wasn't alchemical luck. The researchers repeated the experiment with a randomly sampled 1,389 problems as a control—the random subset scored 8.1% lower on average than the full dataset.

The key isn't "less"—it's "choosing right."

---

2. The Big-Data Dogma: AI's Most Stubborn Assumption ⛪

> Scaling Laws: The empirical规律 proposed by OpenAI in 2020, stating that model performance improves predictably with parameters, data, and compute. It fueled the "bigger is better" belief behind the GPT-3 → GPT-4 arms race.

Scaling laws are correct, but they have a dangerous side effect: people mistake "scale" for "essence."

For five years, the AI industry has run on an unspoken rule: data volume × compute = intelligence. 100K RL samples beat 10K; 1M beats 100K. PRIME used 150K samples, Sky-T1 used 45K, DeepScaleR used 40K—the numbers keep getting scarier.

But nobody asked: is every one of those 150K samples actually teaching the model something useful?

The LIMR paper dissects this assumption like a scalpel. The researchers found: in RL training, most data is noise—it doesn't drive learning, it just occupies GPUs.

> Noisy Data: In machine learning, samples that contribute nothing to the learning objective or actively interfere with optimization. In the RL context, some problems the model already masters (zero reward growth), others it will never solve (reward constant at zero). They fill training batches without changing model behavior.

Ironically, randomly discarding 84% of the data hurts (RAND baseline dropped 8.1%). But selecting the right 16% with the right method makes the model not just equal but stronger.

This reveals something brutal: large datasets have extremely low value density. Like a gold mine where 80% of rocks contain no gold—but you haul them all to the surface anyway, because no one tells you which rocks hold gold.

---

3. LIM: How Do You Compute a Sample's "Learning Value"? 🔬

The method in the paper is called Learning Impact Measurement (LIM). Its core idea is simple:

> "The best data is the data that most helps the model's learning trajectory."

LIM doesn't care how hard a problem is, how long the answer is, or where it came from. It looks at one thing: how "in sync" is this problem's reward curve with the model's overall reward curve during training?

Specifically, the researchers recorded each problem's reward $$r_i^k$$ (problem $$i$$ at training round $$k$$ ), then computed the batch's average reward curve:

r_{\text{avg}}^k = \frac{1}{N} \sum_{i=1}^{N} r_i^k

> Average Reward Curve: The trajectory of mean reward across all samples over training rounds. It reflects the model's overall improvement path—typically logarithmic: fast early, slow late.

Next, LIM scores each problem by how similar its own reward trajectory is to this average line:

s_i = 1 - \frac{\sum_{k=1}^{K}(r_i^k - r_{\text{avg}}^k)^2}{\sum_{k=1}^{K}(1 - r_{\text{avg}}^k)^2}

> LIM Score: Range $(-\infty, 1]$ . Higher scores mean the sample's learning trajectory "resonates" with the model's overall learning rhythm. The numerator is the mean squared error between the sample trajectory and the average trajectory; the denominator is a normalization factor.

The intuition:

If a problem's rewards rise and fall with the model's overall progress → high LIM score → it's "teaching the model how to learn"
If a problem is always zero (never learned) or always full marks (already mastered) → low LIM score → no teaching value

Setting a threshold of

\theta = 0.6

, they filtered 8,523 problems down to 1,389—the LIMR dataset.

> Threshold Filtering: A hard cutoff keeping only samples above a score threshold. $\theta=0.6$ retains samples highly aligned with the model's learning trajectory.

---

4. Why RL Suits the "Fine Grains" Strategy Better Than SFT ⚔️

One finding struck the author as even more shocking than data compression itself:

On a 7B model, RL + curated data crushed SFT + distilled data.

| Method | Data | AIME24 | MATH500 | AMC23 | Avg | |:----:|:------:|:------:|:-------:|:-----:|:------:| | LIMR (RL) | 1,389 | 32.5% | 78.0% | 63.8% | 58.1% | | LIMO (SFT) | 817 | 15.8% | 65.0% | 56.3% | 45.7% | | s1 (SFT) | 1,000 | 15.8% | 55.8% | 42.5% | 38.0% |

> SFT (Supervised Fine-Tuning): Training a model to imitate high-quality reference answers. LIMO and s1 both generate long chain-of-thought answers with a large model (e.g., 32B), then fine-tune a small model on them—distillation. > > RL (Reinforcement Learning): The model attempts problems itself, gets rewards based on correctness, and finds optimal strategies by trial and error—not imitation.

Same 7B model, same "small data" setting—yet LIMR more than doubled LIMO's AIME24 score (32.5% vs 15.8%).

What does this mean? Imitating experts doesn't necessarily make you stronger, but practicing the problems that work for you, through your own trial and error, does.

SFT's problem: it force-feeds a small model "standard answers" generated by a big model. If those answers exceed the small model's comprehension—like making a middle schooler copy a college student's calculus notes—perfect imitation isn't real understanding.

RL's advantage: the model only practices problems it can reach with effort. Too easy (already full marks) leaves no reward headroom; too hard (always zero) gives no positive feedback. LIM essentially automates finding the optimal training zone.

> Zone of Proximal Development: An educational psychology concept—the range of tasks a learner can master with guidance but not yet alone. LIM effectively identifies this zone algorithmically.

---

5. The Most Uncomfortable Implication 😰

Teams that spent millions annotating 100K+ datasets may have wasted 80% of their money.

Worse, that "garbage data" may not just be useless—it may be slowing training down. Figure 3(c) of the paper shows LIMR's reward curve rising faster and more stably, while full-data training oscillates noticeably early on. Low-value samples don't just idle GPUs; they can steer the model astray.

> Training Instability: In RL, when batch sample quality varies wildly, policy gradient directions fluctuate sharply, producing inconsistent parameter updates. Low-value samples in full datasets can generate misleading gradients and delay convergence.

A broader worry: amid the current wave of open-source o1/R1 reproduction efforts, countless teams are frantically collecting, annotating, and cleaning datasets, trying to compensate for quality with quantity. But if LIMR's findings generalize—the industry may be collectively burning money on useless work.

Caveats: LIMR was validated only on math reasoning, with a 7B model. Whether 32B+ models need the same filtering, and whether it applies to code, science, or multimodal domains, the paper doesn't answer.

But even so, 1,389 beating 8,523 is a number that can keep you up at night.

---

6. The Bet 💎

My judgment: the big-data dogma in RL training is collapsing; data quality matters about 6× more than quantity. Teams still selling "we used 150K samples" are the same crowd that once bragged about "175B parameters."

The enemy I'm naming: the industry's unspoken rule that "data volume = effort = model quality." It's false, it's popular, and it makes annotation companies and cloud vendors laugh all the way to the bank.

The price I pay: if LIMR's findings are a math-only peculiarity or only hold at 7B, I'm peddling a "small-sample hallucination," misleading resource-constrained teams into abandoning scale. I'll bear that reputation.

But I still bet on quality. Real learning isn't "how many problems you drilled" but "which problems changed you." An athlete who practiced 1,389 key movements can beat one who practiced 8,523 random ones.

AI is no exception.

> Next time someone says "our RL dataset has 100K samples," don't ask "that many?"—ask "how many are actually teaching the model something new?" 💡

---

📚 Paper Details (Verified)

| Item | Content | |:----:|:-----| | Title | LIMR: Less Is More for RL Scaling | | Authors | Xuefeng Li, Haoyang Zou, Pengfei Liu | | Institutions | SJTU / SII / GAIR | | arXiv ID | 2502.11886 | | Published | February 17, 2025 | | Code/Data | https://github.com/GAIR-NLP/LIMR | | Core Method | Learning Impact Measurement (LIM): score samples by alignment of reward trajectory with the model's average reward curve; filter high-value samples | | Model | Qwen2.5-Math-7B | | Algorithm | PPO (OpenRLHF framework) | | Training Data | MATH-FULL (8,523 problems, Level 3-5) → filtered to LIMR (1,389, $\theta=0.6$ ) | | Benchmarks | MATH500, AIME2024, AMC2023 | | Key Result | LIMR (1,389): 78.0% on MATH500 vs 76.6% full; tied 32.5% on AIME24; 63.8% vs 61.9% on AMC23. Over 100% AIME24 improvement vs SFT methods LIMO/s1 | | Reward Design | Correct: +1; wrong but well-formatted: -0.5; malformed: -1 |