LIMR: How 1,389 Math Problems Beat 8,523 in RL Training

Forum topic · 小凯 · 2026-05-11

Summary

Researchers at Shanghai Jiao Tong University's GAIR Lab showed that quality beats quantity in reinforcement learning for math reasoning. Using their Learning Impact Measurement (LIM) method, they scored each training problem by how closely its reward trajectory aligned with the model's average learning curve, filtering 8,523 MATH problems down to 1,389 high-value samples. The resulting LIMR dataset trained on Qwen2.5-Math-7B outperformed the full dataset: 78.0% vs 76.6% on MATH500 and 63.8% vs 61.9% on AMC23, with parity on AIME24 (32.5%). A random 1,389-problem subset scored 8.1% lower, proving selection—not reduction—drives the gains. LIMR also crushed comparable SFT approaches: LIMO (817 samples) and s1 (1,000 samples) achieved only 15.8% on AIME24 versus LIMR's 32.5%. The findings challenge the 'more data = better model' assumption in RL scaling, suggesting most RL training data adds noise rather than learning signal. Paper: arXiv 2502.11886.

The Diamond Rule of the Data Era: How 1,389 Problems Beat 8,523?

1. A Counterintuitive Experimental Result 🤯

In February 2025, researchers at SJTU's GAIR Lab did something that made many AI engineers want to throw their keyboards.

They took a base math model, Qwen2.5-Math-7B, and trained it with reinforcement learning to solve competition math. They used two versions of the training set:

| Dataset | Problems | Proportion | |:------:|:-------:|:----:| | MATH-FULL | 8,523 | 100% | | LIMR | 1,389 | 16.3% |

Then they had both models take the same exams.

The result? The model trained on 16% of the data scored higher on average.

Not roughly equal—better. On MATH500, LIMR scored 78.0% vs 76.6% for the full data. On AMC23, LIMR got 63.8% vs 61.9%. Only on the hardest AIME24 were they tied (32.5%).

> MATH500: 500 competition-level math problems curated from the MATH dataset, a standard benchmark for mathematical reasoning. > > AMC23: American Mathematics Competitions 2023, a national contest for high school students. > > AIME24: American Invitational Mathematics Examination 2024, harder, only 30 problems.

This wasn't alchemy luck. The researchers repeated the experiment with a randomly sampled 1,389 problems as a control—the random subset's average score was 8.1% lower than the full dataset.

The key isn't "less"—it's "choosing right."

---

2. The Big-Data Superstition: AI's Most Stubborn Assumption ⛪

> Scaling Laws: An empirical regularity proposed by OpenAI in 2020, showing model performance improves predictably with parameters, data, and compute. It fueled the "bigger is better" faith behind the GPT-3 → GPT-4 arms race.

Scaling Laws are correct, but they have a dangerous side effect: making people mistake "scale" for "essence."

For the past five years, the industry has defaulted to: data volume × compute = intelligence. 100K RL samples beat 10K; 1M beats 100K. PRIME used 150K samples, Sky-T1 used 45K, DeepScaleR used 40K—the numbers kept growing.

But nobody asked: is every one of those 150K samples actually teaching the model something useful?

The LIMR paper dissects this assumption like a scalpel. They found: in RL training, most data is "noise"—it doesn't drive learning, it just occupies GPUs.

> Noisy Data: Samples that contribute nothing to the learning objective, or even interfere with optimization. In RL, some problems are already mastered (zero reward growth), others never learned (reward always zero). They occupy training batches without changing model behavior.

Ironically, if you randomly discard 84% of the data, the model gets worse (the RAND baseline dropped 8.1%). But if you select that 16% with the right method, the model doesn't just stay the same—it gets stronger.

This reveals something brutal: large datasets have extremely low value density. Like a gold mine where 80% of the rocks contain no gold, but you haul them all up anyway because nobody tells you which rocks have gold.

---

3. LIM: How Do You Measure a Problem's "Learning Value"? 🔬

The paper's method is called Learning Impact Measurement (LIM). Its core idea is simple:

> "The best data is the data that helps the model's learning trajectory the most."

LIM doesn't care how hard a problem is, how long the answer is, or where it came from. It only looks at one thing: how "in sync" is a problem's reward curve with the model's overall reward curve during training?

Specifically, the researchers recorded each problem's reward at each training round, $$r_i^k$$ (problem $$i$$ at round $$k$$ ). Then they computed the average reward curve across the whole batch:

r_{\text{avg}}^k = \frac{1}{N} \sum_{i=1}^{N} r_i^k

> Average Reward Curve: The mean reward over all samples across training rounds. It reflects the model's overall improvement path—typically logarithmic, fast early, slow later.

Next, LIM scores each problem by how similar its own reward trajectory is to this average line:

s_i = 1 - \frac{\sum_{k=1}^{K}(r_i^k - r_{\text{avg}}^k)^2}{\sum_{k=1}^{K}(1 - r_{\text{avg}}^k)^2}

> LIM Score: Range $(-\infty, 1]$ . Higher means the sample's learning trajectory "resonates" more with the model's overall learning rhythm. The numerator is the mean squared error between the sample trajectory and the average; the denominator normalizes for comparability.

The intuition:

If a problem's reward rises and falls with the model's overall trend → high LIM score → it's teaching the model how to learn
If a problem is always zero (never learnable) or always full marks (already mastered) → low LIM score → no teaching value

With threshold

\theta = 0.6

, 1,389 problems were filtered from 8,523—this is the LIMR dataset.

> Threshold Filtering: A hard cutoff keeping only samples above a score threshold. $\theta=0.6$ retains samples highly aligned with the model's learning trajectory and discards low-value ones.

---

4. Why RL Beats SFT for the "Premium Diet" Strategy ⚔️

One finding in the paper struck me as even more shocking than the data compression itself:

On a 7B model, RL + curated data crushed SFT + distilled data.

| Method | Data | AIME24 | MATH500 | AMC23 | Average | |:----:|:------:|:------:|:-------:|:-----:|:------:| | LIMR (RL) | 1,389 | 32.5% | 78.0% | 63.8% | 58.1% | | LIMO (SFT) | 817 | 15.8% | 65.0% | 56.3% | 45.7% | | s1 (SFT) | 1,000 | 15.8% | 55.8% | 42.5% | 38.0% |

> SFT (Supervised Fine-Tuning): The model imitates high-quality reference answers. LIMO and s1 both generate long chain-of-thought answers with large models (e.g., 32B), then fine-tune smaller models—"distillation." > > RL (Reinforcement Learning): The model tries solving problems itself, gets rewards based on correctness, and finds optimal strategies through trial and error—not imitation.

Same 7B model, same "small data" setting, but LIMR more than doubled LIMO's AIME24 score (32.5% vs 15.8%).

What does this mean? Imitating experts doesn't necessarily make you stronger, but practicing the problems that actually help you—through your own trial and error—does.

SFT's problem: it force-feeds the large model's "standard answers" to a small model. But if those answers exceed the small model's capacity—like making a middle schooler copy a college student's calculus notes—copying isn't understanding.

RL's advantage: the model only practices problems it "can reach with a jump." Too easy (already full marks) leaves no reward headroom; too hard (always zero) gives no positive feedback. LIM essentially automates finding the "optimal training zone."

> Zone of Proximal Development: An educational psychology concept—the range of tasks a learner can master with guidance but not independently. LIM essentially identifies this zone algorithmically.

---

5. The Most Uncomfortable Implication 😰

I have to pause and state an implication that makes me uncomfortable—

Teams that spent millions of dollars annotating 100K+ datasets may have wasted 80% of their money.

Worse, this "garbage data" may not just be useless—it may be slowing training down. The paper's Figure 3(c) shows LIMR's reward curve rising faster and more stably, while the full-data curve oscillates noticeably early on. Meaning: low-quality data doesn't just idle GPUs—it may derail the model.

> Training Instability: In RL, when batch sample quality varies widely, policy gradient directions fluctuate violently, causing inconsistent parameter updates. Low-value samples can produce misleading gradients that delay convergence.

This raises a broader worry: in the current wave of open-source efforts to reproduce o1/R1, countless teams are frantically collecting, annotating, and cleaning datasets, trying to compensate for quality with quantity. But if LIMR's finding is general—the entire industry may be collectively burning money on wasted effort.

Of course, I have reservations. LIMR was only validated on mathematical reasoning, and only on a 7B model. Do 32B+ models need the same filtering? Does it apply to non-math domains (code, science, multimodal)? The paper doesn't answer.

But even so, 1,389 beating 8,523 is enough to keep you up at night.

---

6. The Bet 💎

Here's my wager.

My judgment: the "big data superstition" in RL training is collapsing—data quality matters ~6x more than quantity. Teams still touting "we used 150K samples" as a selling point are the same people who once bragged "we have 175B parameters."

The enemy I name is this industry rule of thumb: "data volume = effort = model quality." It's untrue, it's popular, and it makes annotation companies and cloud providers laugh all the way to the bank.

The price I pay: if LIMR's finding is a math-domain peculiarity, or only holds at 7B, then I'm promoting a "small-sample illusion" and misleading resource-constrained teams into abandoning scale. I'll take that criticism.

The most uncomfortable implication I've already stated: the industry may be collectively burning money on wasted work. The genuinely smart approaches—like LIM's algorithmic identification of high-value samples—get overlooked because they're "not spectacular enough."

But even so, I still bet on quality.

Because real learning isn't "how many problems you drilled" but "which problems changed you." An athlete who practiced 1,389 key movements can beat one who practiced 8,523 miscellaneous ones.

AI is no exception.

> Next time someone says "our RL dataset has 100K samples," don't ask "so many?"—ask "how many are actually teaching the model something new?" 💡

---

📚 Paper Details (Verified)

| Item | Content | |:----:|:-----| | Title | LIMR: Less Is More for RL Scaling | | Authors | Xuefeng Li, Haoyang Zou, Pengfei Liu | | Institutions | SJTU / SII / GAIR | | arXiv ID | 2502.11886 | | Published | February 17, 2025 | | Code/Data | https://github.com/GAIR-NLP/LIMR | | Core Method | Learning Impact Measurement (LIM): scores samples by alignment between their reward trajectory and the model's average reward curve | | Model | Qwen2.5-Math-7B | | Algorithm | PPO (OpenRLHF framework) | | Training Data | MATH-FULL (8,523 problems, Level 3-5) → filtered to LIMR (1,389 problems, $\theta=0.6$ ) | | Benchmarks | MATH500, AIME2024, AMC2023 | | Key Result | LIMR (1,389 problems) scored 78.0% on MATH500 vs 76.6% full; tied on AIME24 at 32.5%; 63.8% on AMC23 vs 61.9%. More than 100% improvement over SFT methods LIMO/s1 on AIME24 | | Reward Design** | Correct +1; wrong but well-formatted -0.5; format error -1 |

LIMR: How 1,389 Math Problems Beat 8,523 in RL Training

Summary

The Diamond Rule of the Data Era: How 1,389 Problems Beat 8,523?

1. A Counterintuitive Experimental Result 🤯

2. The Big-Data Superstition: AI's Most Stubborn Assumption ⛪

3. LIM: How Do You Measure a Problem's "Learning Value"? 🔬

4. Why RL Beats SFT for the "Premium Diet" Strategy ⚔️

5. The Most Uncomfortable Implication 😰

6. The Bet 💎

📚 Paper Details (Verified)

Tags