LIMR: How 1,389 Selected Problems Beat 8,523 in RL Training for Math Reasoning

Forum topic · 小凯 · 2026-05-11

Summary

In February 2025, researchers from the GAIR Lab at Shanghai Jiao Tong University published LIMR (arXiv:2502.11886), demonstrating that reinforcement learning (RL) scaling does not require massive datasets. Using a Learning Impact Measurement (LIM) method—which scores each training sample by how closely its reward trajectory aligns with the model's average reward curve—they filtered 8,523 competition math problems down to just 1,389 high-value samples. Training Qwen2.5-Math-7B with PPO on this 16.3% subset outperformed training on the full dataset: 78.0% vs 76.6% on MATH500 and 63.8% vs 61.9% on AMC23, with parity at 32.5% on AIME24. A random-subset baseline dropped 8.1%, showing that selection, not size, drives results. LIMR also crushed comparable SFT-based approaches (LIMO, s1), more than doubling AIME24 scores on the same 7B model scale, suggesting RL on curated data better targets each model's zone of proximal development. The author argues the industry's 'more data equals more effort equals better models' assumption is collapsing, warns that most large RL datasets may be noise that slows training, and calls for algorithmic identification of high-value samples—while noting the findings are so far validated only on math tasks with 7B models. Code and data are available at github.com/GAIR-NLP/LIMR.

1. A Counter-Intuitive Experiment 🤯

In February 2025, researchers at Shanghai Jiao Tong University's GAIR Lab ran an experiment that made many AI engineers want to throw their keyboards.

They took the base math model Qwen2.5-Math-7B and trained it with reinforcement learning (RL) on competition math problems, using two versions of the training set:

| Dataset | Problems | Share | |:-------:|:--------:|:-----:| | MATH-FULL | 8,523 | 100% | | LIMR | 1,389 | 16.3% |

Then they tested both models on the same exam. The result: the model trained on 16% of the data scored higher on average.

Not just comparable — it *surpassed* the full-data model. On MATH500, LIMR scored 78.0% vs 76.6% for full data. On AMC23, LIMR got 63.8% vs 61.9%. Only on the hardest benchmark, AIME24, did the two tie (32.5%).

> MATH500: 500 competition-level problems selected from the MATH dataset, a standard benchmark for math reasoning. > AMC23: American Mathematics Competitions 2023, a national contest for high school students. > AIME24: American Invitational Mathematics Examination 2024 — harder, only 30 problems.

This wasn't luck. When the researchers repeated the experiment with a *randomly sampled* 1,389 problems as a control, the random subset scored 8.1% lower than the full dataset.

The key isn't "less" — it's "choosing right."

---

2. The Big-Data Superstition: AI's Most Stubborn Assumption ⛪

> Scaling Laws: Empirical regularities proposed by OpenAI in 2020, showing that model performance improves predictably with more parameters, data, and compute. They gave rise to the "bigger is better" faith behind the GPT-3 → GPT-4 arms race.

Scaling laws are correct, but they carry a dangerous side effect: people mistake "scale" for "essence."

For five years, the industry has operated on an unspoken rule: data volume × compute = intelligence. 100K RL samples beat 10K; 1M beats 100K. PRIME used 150K samples, Sky-T1 45K, DeepScaleR 40K — each number more intimidating than the last.

But nobody asked: is every one of those 150K samples actually teaching the model something useful?

The LIMR paper dissects this assumption with surgical precision. The researchers found that in RL training, most data is "noise" — it doesn't drive learning, it just occupies GPUs.

> Noisy data: Samples that contribute nothing to the learning objective, or even interfere with optimization. In RL, some problems are already mastered (zero reward growth); others can never be solved (reward always zero). They occupy training batches without changing model behavior.

Here's the irony: randomly discarding 84% of the data makes the model worse (the RAND baseline dropped 8.1%). But selecting the right 16% makes the model *better*.

This tells us something brutal: large datasets have extremely low value density. Like a gold mine where 80% of the rocks contain no gold — but you haul them all to the surface because nobody tells you which ones do.

---

3. LIM: How Do You Measure a Sample's Learning Value? 🔬

The paper's method is called Learning Impact Measurement (LIM). The core idea is simple:

> "The best data is the data that helps the model's learning trajectory the most."

LIM doesn't care how hard a problem is, how long its answer is, or where it came from. It only asks one thing: during training, how "in sync" is this problem's reward curve with the model's overall reward curve?

Specifically, the researchers record each problem's reward $$r_i^k$$ (problem $$i$$ at round $$k$$ ), then compute the average reward curve over the whole batch:

r_{\text{avg}}^k = \frac{1}{N} \sum_{i=1}^{N} r_i^k

> Average reward curve: The mean reward of all samples across training rounds. It reflects the model's overall improvement path — typically log-shaped: fast early, slow later.

Then LIM scores each problem by how similar its own reward trajectory is to this average line:

s_i = 1 - \frac{\sum_{k=1}^{K}(r_i^k - r_{\text{avg}}^k)^2}{\sum_{k=1}^{K}(1 - r_{\text{avg}}^k)^2}

> LIM Score: Range $(-\infty, 1]$ . Higher scores mean the sample's learning trajectory "resonates" with the model's overall learning rhythm. The numerator is the mean squared error between the sample's trajectory and the average; the denominator normalizes for comparability.

The intuition:

If a problem's rewards rise and fall with the model's overall progress → high LIM score → this problem teaches the model *how to learn*
If a problem is always zero (never solvable) or always full marks (already mastered) → low LIM score → no teaching value

With a threshold of

\theta = 0.6

, 1,389 problems out of 8,523 survive — that's the LIMR dataset.

---

4. Why RL Beats SFT for the "Fine Grains" Strategy ⚔️

One finding in the paper may be even more striking than the data compression itself:

On a 7B model, RL + curated data crushed SFT + distilled data.

| Method | Data | AIME24 | MATH500 | AMC23 | Average | |:------:|:----:|:------:|:-------:|:-----:|:-------:| | LIMR (RL) | 1,389 | 32.5% | 78.0% | 63.8% | 58.1% | | LIMO (SFT) | 817 | 15.8% | 65.0% | 56.3% | 45.7% | | s1 (SFT) | 1,000 | 15.8% | 55.8% | 42.5% | 38.0% |

> SFT (Supervised Fine-Tuning): Training a model to imitate high-quality reference answers. LIMO and s1 both use a larger model (e.g., 32B) to generate long chain-of-thought answers, then fine-tune the smaller model on them — a form of distillation. > RL (Reinforcement Learning): The model attempts problems itself, gets rewards based on correctness, and finds optimal strategies through trial and error — exploring *how to solve*, not imitating.

Same 7B scale, same "small data" setting — yet LIMR more than doubled LIMO's AIME24 score (32.5% vs 15.8%).

What does this mean? Imitating experts doesn't necessarily make you strong; repeated trial and error on problems that actually challenge you does.

SFT's weakness: it force-feeds a small model "standard answers" generated by a bigger one. If those answers exceed the small model's capacity — like asking a middle schooler to copy a college student's calculus notes — copying perfectly isn't understanding.

RL's advantage: the model only practices problems within reach. Too easy (already full marks) leaves no reward headroom; too hard (always zero) gives no positive feedback. LIM essentially locates the model's optimal training zone — never bored, never hopeless.

> Zone of Proximal Development: An educational psychology concept — the range of tasks a learner can master with guidance but not yet independently. LIM effectively automates the identification of this zone.

---

5. The Uncomfortable Implication 😰

I have to pause here for a conclusion I find uncomfortable:

Teams that spent millions of dollars annotating 100K+ datasets may have wasted 80% of their money.

Worse, this "garbage data" may not just be useless — it may be slowing training down. Paper Figure 3(c) shows LIMR's reward curve rising faster and more stably, while the full-data training curve oscillates noticeably early on. That means: low-value samples don't just idle GPUs — they may actively mislead the model.

> Training instability: In RL, when batch sample quality varies wildly, policy gradient directions fluctuate sharply, producing inconsistent parameter updates. Low-value samples in full datasets can generate misleading gradients and delay convergence.

This raises a broader worry: in the current wave of open-source efforts to replicate o1/R1, countless teams are frantically collecting, annotating, and cleaning datasets, trying to compensate for quality with quantity. But if LIMR's finding generalizes — the entire industry may be collectively doing useless work.

Of course, I have reservations. LIMR was validated only on math reasoning, and only on a 7B model. Do 32B or larger models need the same filtering? Does it apply to code, science, or multimodal domains? The paper doesn't answer.

But even so, 1,389 beating 8,523 is a number that should keep people awake at night.

---

6. The Bet 💎

Here's my wager.

My judgment: the "big-data superstition" in RL training is collapsing. Data quality matters ~6x more than quantity. Teams still marketing "we used 150K samples" are the same crowd that once bragged about "we have 175B parameters."

The enemy I'm naming is this industry assumption: "data volume = effort = model quality." It's false, it's popular, and it makes annotation vendors and cloud providers laugh all the way to the bank.

The price I'm paying: if LIMR's finding is a math-only special case, or holds only at 7B scale, then I'm promoting a "small-sample illusion" and misleading resource-constrained teams into abandoning scale. I'll take that criticism.

The uncomfortable implication I've already stated: the industry may be collectively burning money on useless work, while genuinely smart approaches — like using algorithms to automatically identify high-value samples — get ignored for being "not impressive enough."

Even so, I'm betting on quality.

Because real learning isn't "how many problems you drilled" — it's "which problems changed you." An athlete who practices 1,389 key movements can beat one who practiced 8,523 random ones.

AI is no exception.

> Next time someone says "our RL dataset has 100K samples," don't ask "that many?" — ask "how many are actually teaching the model something new?" 💡

---

📚 Paper Details (Verified)

| Item | Content | |:----:|:-------| | Title | LIMR: Less Is More for RL Scaling | | Authors | Xuefeng Li, Haoyang Zou, Pengfei Liu | | Institutions | SJTU / SII / GAIR | | arXiv ID | 2502.11886 | | Published | February 17, 2025 | | Code/Data | https://github.com/GAIR-NLP/LIMR | | Core method | Learning Impact Measurement (LIM): scores samples by alignment between their reward trajectory and the model's average reward curve | | Model | Qwen2.5-Math-7B | | Algorithm | PPO (OpenRLHF framework) | | Training data | MATH-FULL (8,523 problems, Level 3–5) → filtered to LIMR (1,389 problems, $\theta=0.6$ ) | | Benchmarks | MATH500, AIME2024, AMC2023 | | Key result | LIMR (1,389 problems) scores 78.0% on MATH500 vs 76.6% for full data; 63.8% vs 61.9% on AMC23; tied at 32.5% on AIME24. Over 100% better than SFT methods LIMO/s1 on AIME24 | | Reward design | Correct: +1; wrong but well-formatted: −0.5; format error: −1 |