A Beginner at the Foot of the Mountain Needs a Coach Who Kneels Down
*Full translation of a zhichai.net forum post (teaching-signal series, part 2).*
🧗 He has fallen on the same rock countless times
The mountain is called "high school math olympiad." A climber stands at its foot, looks up, and sees the route written clearly on the rock face: a topology problem — prove whether a subset of a nowhere dense set can be non-Borel. He takes a deep breath and starts climbing.
The first time, he slips off at the third hold. The second time, he takes a different route but slides down at the same spot. The third and fourth times, he falls on the same rock in exactly the same posture. Each time he leaves a muttering on the wall — "the answer should be No" — and drops back to the ground.
There are three kinds of coaches.
The first coach says: "This wall isn't for you; let's climb another mountain." He takes you away and swaps in an easier problem. You no longer fall, but you never learn where that fall went wrong. This is filtering.
The second coach says: "Here, watch the pre-game tape." He hands you a stack of notes full of climbing tips summarized by predecessors. Those notes were written three months ago, when your grip, stride, and way of reading the wall were all different. Some advice still helps; some has started to mislead you. This is static curricula.
The third coach says: "Let's review the footage tonight." He records your fall and replays it frame by frame, telling you "here you should have planted your foot differently." You nod and remember the correct answer. But facing a new wall, your hands still don't know where to go. This is post-hoc repair.
The paper asks: can a coach kneel down at the exact moment of every fall, look you in the eye, and say — "your problem isn't your legs, it's how you read the route"?
This is MENTOR: Learning from Failure with On-Policy Pedagogical Intervention, proposed by a Xi'an Jiaotong University team. It installs an "on-site tutor" into reinforcement learning training. Before discussing it, we must see why the climber is stuck at the foot of the mountain.
📉 Zero-reward collapse: when an entire batch of attempts fails
Over the past two years, LLM post-training has shifted from imitating human answers (SFT) to solving and self-scoring (RLVR, Reinforcement Learning with Verifiable Rewards). The typical representative is GRPO (Group Relative Policy Optimization) from DeepSeek: give the model a problem with a standard answer, sample a group of solutions (say, four), score each with rules — 1 for correct, 0 for wrong — then compare within the group: better-than-peers trajectories are encouraged, worse ones suppressed. It's like sending four climbers up the wall and imitating whoever climbs highest.
GRPO's cleverness: no reward model needed, saving training cost and avoiding judge-model preference contamination.
But GRPO has a structural weakness the paper calls zero-reward collapse.
Imagine a problem beyond the model's ability: four climbers all fail, all rewards are 0. How do you compute "relative advantage"? All-correct groups get advantage 0; all-wrong groups also get 0 — no trajectory is better or worse than its peers. Zero advantage, zero gradient, zero contribution to the update. The model learns from this group as much as from blank paper: nothing.
Worse, it's self-reinforcing: too hard → all wrong → nothing learned → model stuck → still too hard. The training curve plateaus like a wall. Ironically, the model's deepest blind spots hide exactly in these all-wrong problems — problems it "almost gets" are naturally separated by group comparison. The collapse site buries the most valuable information, and the standard pipeline ignores it.
The community has noticed, but the three existing families of fixes each carry a scar.
🩹 Three old fixes, three old wounds
Family 1: Filtering avoidance. DAPO discards zero-gradient groups. Crude but effective for stability — yet defeatist: those failure trajectories are full of evidence of what the model "thought it was doing right," the most valuable diagnostic material, thrown straight into the trash. Swap mountains and you won't fall again — but you never learned to climb.
Family 2: Static offline guidance. LUFFY, QUESTA, HINT, Scaf-GRPO prepare prompt libraries, scaffolds, or curricula ahead of time. But this is off-policy — the hints are made before training while the model keeps evolving. Failure modes at step 100 differ completely from step 1000; preset hints become misaligned, even counterproductive. The tape you watched before the game was for the you of three months ago; the fall you take on the field is today's fall.
Family 3: Post-hoc repair. Critique-GRPO, SRPO, CTRL reflect on, refine, and correct failed trajectories, using "fixed answers" for data augmentation. Better than filtering — at least failure information is used. But its philosophy: it repairs data, it doesn't teach the student. Correcting an answer is re-grading the exam for the student; the missing skill — "what to think when seeing a nowhere dense set" — was never taught. Retrospective patching cannot replace prospective teaching.
MENTOR's stance is thus clear: we need on-policy pedagogical intervention — an on-site tutor that diagnoses the root cause of failure in real time, dynamically generates targeted teaching hints, and guides the next attempt. Not pre-class preview, not post-class commentary, but immediate tutoring at the moment of getting stuck.
Behind this lies an old pedagogical concept: Vygotsky's Zone of Proximal Development. The height a student can't reach alone, they can reach with guidance. A good teacher's skill isn't giving answers but providing precisely calibrated scaffolding — give a little, let the student build the rest. MENTOR aims to fit this kind of teaching into the RL training loop.
🏥 Tiered care: MENTOR's three clinics
MENTOR's architecture resembles hospital tiered care: a frontline doctor screens critical patients (CASF), specialists consult and prescribe with a pharmacist reviewing (the FDCG four-agent pipeline), then the patient returns to the regular ward (integration back into GRPO).
🚨 First clinic: CASF, screening "collapse patients"
CASF (Collapse-Aware Sample Filtering) is simple: a hook in the training loop watches each group's rewards. If all N trajectories of a problem score zero, the problem is flagged into a collapse set C.
But CASF isn't DAPO-style "throw it away." It preserves the full failure scene: problem, standard answer, all N failed trajectories, packed as a triple — like an ER keeping complete medical records and surveillance footage, not to heal the patient now, but for the later consultation.
🔬 Second clinic: FDCG, a four-agent consultation room
The core module is FDCG (Failure-Driven Contextual Guidance), a four-agent teaching pipeline with Qwen3-32B playing the tutor.
Attribution Agent: the attending physician. Given the problem, answer, and all failure trajectories, it performs root-cause diagnosis — not which step of one trajectory failed, but the persistent error shared across trajectories. In the topology example, all four trajectories answer "No"; the diagnosis: "confused the relation between nowhere dense sets and F_σ sets."
Design principle: attribution locates the cognitive gap but never leaks the solution path. It tells you where the disease is, not how to mix the medicine. That's the line between teaching and feeding.
Generation Agent: uses the diagnosis keywords as instructions and failure trajectories as context to produce candidate teaching hints. Deliberate constraint: avoid leaking answers or too many reasoning steps. The hint scaffolds so the model rediscovers the solution itself. In the topology case, the hint is roughly: "Think — is every nowhere dense set an F_σ set?..." — pointed, no more.
Judgment-Repair Loop: quality assurance. The Judgment Agent evaluates hints via rules plus LLM-as-Judge, rating LOW/HIGH. In one real case, the first hint was judged LOW ("too vague"); the Repair Agent iterated a more directive version, judged HIGH, passed. If repairs repeatedly fail, the system falls back to a preset safe hint — mediocre is acceptable; toxic is not.
The loop embodies a plain pedagogical truth: hint quality beats hint quantity — one directive-but-non-leaking hint beats ten vague ones. And quality must be audited, not used as generated.
💊 Third clinic: back to GRPO, prescription takes effect
Finally, the validated hint h* is prepended to the original problem, forming q' = h* ⊕ q, which re-enters standard GRPO sampling. The scaffold is up; the climber returns to the wall — and now, with the nudge about nowhere dense sets and F_σ sets, the policy is likelier to produce non-zero-reward trajectories. The gradient returns; learning continues.
Crucially: GRPO's loss, optimizer, and reward mechanism are untouched. MENTOR wraps a pedagogical outpost around the pipeline — plug and play. Total loss is just GRPO on normal data plus GRPO on augmented data.
📊 Experiments: five models, eight tracks, one-sided gains
The evaluation covers 5 models across math reasoning (AIME24/25, MATH-500, GAOKAO-Bench, Olympiad-Bench, DM), code generation (LCB, HumanEval, MBPP, BigCodeBench), and out-of-distribution testing (MMLU), against 8 baselines: GRPO, DAPO, SRPO, Critique-GRPO, QUESTA, LUFFY, HINT, Scaf-GRPO.
Selected numbers:
- DeepSeek-R1-Distill-Qwen-1.5B (math average): 38.57 → 52.16, +35.2%; code 17.17 → 24.65, +43.6%.
- Llama-3.2-3B-Instruct (math average): 18.57 → 33.11, +78.3%. AIME24 +700.9% and GAOKAO +1674.2% are small-base effects — but the base doubling itself is real.
- Qwen3-4B (math average): +10.1%, with AIME25 from 53.33% to 70.00% — a high-score-range gain, worth more because each point near the peak costs more.
- Qwen2.5-Math-7B (math average): +50.0%.
- − Attribution Agent: −4.50%. Hints without diagnosis are blind guesses; teaching loses its target.
- − Generation Agent: −4.67%, the largest drop — counterintuitive and worth pondering. Using the diagnosis keywords directly as hints is the worst option, because diagnosis reports are written in teacher language ("confused nowhere dense sets with F_σ sets"), not student language. From "knowing where the disease is" to "saying something that awakens the student" lies a pedagogical translation — exactly the Generation Agent's job.
- − Judgment-Repair loop: −2.28%. Smallest drop, but it guards the floor: without auditing, one leaking or too-vague hint's damage gets systematically amplified.
- Yue, G., Yu, H., Huang, X., Li, R., Zhang, W., Liu, B., & Wang, R. (2026). MENTOR: Learning from Failure with On-Policy Pedagogical Intervention. arXiv:2610.13594.
- Shao, Z., et al. (2024). DeepSeekMath (GRPO).
- Yu, Q., et al. (2026). DAPO.
- Yan, Y., et al. (2026). LUFFY.
- Zhang, X., et al. (2025a). Critique-GRPO.
- Wan, Z., et al. (2026). SRPO.
- Zhang, X., et al. (2025b). Scaf-GRPO.
- Rafailov, R., et al. (2023). Direct Preference Optimization.
Most telling to the author: MMLU (OOD) also improves consistently. If gains came only from drilling these problems, OOD shouldn't move. It moved — meaning the model learned transferable reasoning structure, not a problem bank. Pedagogically: "learning how to learn."
By contrast, the 8 baselines either abandon failure information (DAPO), suffer distribution drift (static guidance), or fix answers without teaching skills (post-hoc repair). MENTOR's margin shows: when AI gets stuck, how and when you deliver teaching signals matters by orders of magnitude.
🧪 Ablation: which part is least dispensable?
On Qwen3-4B math tasks:
Conclusion: all three parts matter, but translating diagnosis into guiding words is the critical step. Knowing where the student errs and making the student understand are two different abilities.
⏱️ Efficiency: no experts hired, no overtime
Human cost: zero. Scaf-GRPO and QUESTA need ~24 expert hours per task to prepare prompts and curricula. MENTOR's teacher is Qwen3-32B, on call 24/7, unpaid.
Wall-clock time: MENTOR 17.9h vs Critique-GRPO 23.5h vs vanilla GRPO 15.3h. MENTOR is ~24% faster than Critique-GRPO and only 2.6h slower than plain GRPO — teaching costs about a tenth of total training time.
Recall the old fixes' costs: filtering wastes information, static guidance wastes labor, repair wastes compute and time. MENTOR achieves more at lower total cost — in engineering, the rarest kind of result.
🌄 Closing: repairing data is re-grading exams; teaching thinking is holding class
Back to the climber at the foot of the mountain.
Filtering takes him to another mountain, static curricula show him three-month-old tapes, post-hoc repair re-grades his exam. MENTOR's coach kneels at the instant of every fall and says: "Your problem isn't your legs — it's how you read the route. Think: is every nowhere dense set an F_σ set?"
He climbs again. This time three of four trajectories reach the top — he correctly constructs a non-Borel subset of the Cantor set. The answer is Yes.
That is the weight of "on-site tutor." When AI repeatedly hits the wall of its capability boundary, how should teaching signals be given? The paper's answer is almost classical: at the collapse site, inside the training loop, in real time, targeted, and just enough. Not answers but direction; not re-graded exams but taught thinking. One well-placed hint beats ten corrected homework assignments.