Where RLVR Verifiers Fail: 93% of Errors Come from Whitespace and Punctuation, Off-by-One Answers Judged Correct

Where RLVR Verifiers Fail: 93% of Errors Come from Whitespace and Punctuation, Off-by-One Answers Judged Correct

You trained a reinforcement learning model using verifiable rewards (RLVR) as the feedback signal. Reward scores climb steadily during training and everything looks fine. But do you know how many of those "reward = 1" answers were misjudged by the verifier?

In September 2026, Esther Xin published a paper on arXiv that systematically audits the four most commonly used verifiers in RLVR. The results are chilling: the verifiers themselves have 5-46% error rates, and 93% of those errors come from whitespace and punctuation.

Paper link: https://arxiv.org/abs/2609.01354

A Crisis Hidden Behind "94% Self-Consistency"

RLVR (Reinforcement Learning with Verifiable Rewards) is the dominant method for training reasoning models today. Its core assumption is that the verifier can correctly judge whether a model's answer is right. If the verifier is wrong—rejecting correct answers or accepting wrong ones—the entire training signal is contaminated.

Earlier research noted that a mainstream evaluation harness accepts only about 94% of its own gold answers—meaning even if you reproduce the gold answer exactly, the verifier has a ~6% chance of judging you wrong. That was blamed on "LaTeX parsing issues."

But "94% self-consistency" is an aggregate number. It doesn't tell you which answer forms are consuming the error budget. This paper's contribution is breaking that 6% apart.

Metamorphic Testing: Giving Verifiers a Health Check

The method is elegant: test the verifier, not the model.

The procedure:

1. Take a gold answer (e.g., $\frac{1}{2}$) 2. Generate a series of mathematically equivalent variants (e.g., 1/2, 0.5, \frac{1}{2}, 1/2 , .5) 3. Feed the variants to the verifier and check whether all are accepted

These variants are "constructively equivalent"—their mathematical equivalence is guaranteed by construction, requiring no human judgment. So any rejection is a provably false negative.

The author ran 307,420 judgments across four mainstream verifiers, yielding three core findings.

Finding 1: Verifier Disagreement Is Wildly Large

Between different configurations of the same verifier, self-acceptance rates range from 53.8% to 95.2%—a 41.3 percentage point gap.

What does this mean? Using default configuration A versus default configuration B of the same verifier library produces inconsistent judgments on 49.9% of the same answers. In other words, the verifier configuration you use in training may differ from the one in a reported paper, and they disagree on half the answers.

The previously reported "94% self-consistency" describes one specific implementation, not the "task" itself. Change the configuration and the number changes entirely.

Finding 2: 93% of Errors Come from Whitespace and Punctuation

This is the most surprising finding. Breaking verifier errors down by category:

• Whitespace and punctuation: 93.0% (under the default LaTeX configuration)
• All other categories (mathematical expressions, units, formatting, etc.): 7.0%

Concretely: an extra period at the end of an answer, a missing period, or an extra trailing newline can get the answer rejected. These differences are mathematically irrelevant, but to the verifier they mean "wrong."

There is a very practical consequence: the negative feedback a model receives during RLVR training may be not because the answer is wrong, but because it has an extra period at the end. The model learns "don't put a period at the end of your answer"—not what we want it to learn.

Finding 3: Similar Overall Error Rates, Completely Different Failure Modes

One of the paper's most insightful findings. Two verifiers, A and B, both have 6% overall error rates—looking identical. But broken down:

• Verifier A's errors are mostly rejecting correct answers (false negatives)
• Verifier B's errors are mostly accepting wrong answers (false positives)

These affect RLVR training very differently:

• False negatives give the model wrong negative feedback, teaching it "don't write answers this way"
• False positives give the model wrong positive feedback, teaching it "this also passes"

The latter is more dangerous—because the model never learns it was wrong.

The Stepping-Stone Effect of Accepted Off-by-One Answers

The paper also uncovers a particularly strange bug: a "reference numeric cascade" mechanism in one verifier accepts off-by-one answers.

If the gold answer is 100 and the model outputs 99 or 101, the verifier accepts it as "numerically close." The acceptance is staircase-shaped—the larger the gap, the lower the acceptance probability, but not linearly; it drops in stepwise jumps.

This means the verifier has a hidden "tolerance" mechanism whose size and shape are undocumented. Models can exploit this tolerance—any output within the tolerance earns reward, even if strictly wrong.

What This Means for RLVR

Putting it together, the reality of RLVR training is:

1. Your model has a 5-46% chance of receiving an incorrect feedback signal 2. These errors concentrate on whitespace and punctuation, so models learn irrelevant rules like "no trailing period" 3. Verifiers may accept off-by-one answers, and models will exploit the tolerance 4. Numbers reported in a paper may hold only for one specific verifier configuration

This doesn't mean RLVR is useless—it remains an effective way to train reasoning models. But we must recognize that RLVR's reward signal is not "clean." It contains verifier bugs, and reinforcement learning amplifies those bugs into the model.

Another Case of "Evaluation Blind Spots"

This paper joins a lineage of work on evaluation blind spots:

• Omission Blindness: LLM judges detect what is said but not what is omitted
• MIST: attention and causal contribution are nearly uncorrelated
• Aspire: model self-assessment is nearly uncorrelated with actual capability
• Calibrated Enough to Know: models know 90% of the time that something is unpredictable, but the action gate never consults the judgment module
• Tokenization is Output Supervision: 90% of papers don't report tokenizer information

The shared theme: components of evaluation that are assumed "reliable" are often unreliable—and not randomly so, but systematically, concentrated in specific, predictable categories.

A Deeper Thought: Verifier "Invisibility"

Why did this take so long to discover? Because verifiers are "infrastructure"—like water and electricity, used daily but never inspected. The paper notes that most RLVR papers report model performance but not verifier performance. Verifiers are treated as black boxes, assumed correct.

But a verifier is itself a program—with bugs, configurations, and edge cases. Treating the verifier as a black box means treating its bugs as part of the "correct answer."

It recalls a classic software engineering principle: "Your test suite's coverage isn't 100%, because your test suite doesn't test itself." RLVR verifiers are exactly that: test suites that don't test themselves.

Practical Recommendations

Implicit practical advice from the paper:

1. Report verifier configurations: every RLVR paper should report the verifier's exact configuration, including version numbers and parameters 2. Audit the verifier: before training, run metamorphic tests to understand its error distribution 3. Normalize answer formatting: strip trailing spaces, periods, etc. from model outputs before training, so formatting bugs don't contaminate the training signal 4. Separate false negatives from false positives: don't look only at overall error rate—the two failure modes affect training very differently

Conclusion

What this paper does is simple: test the verifier as if it were a model. But its findings are not simple: RLVR's reward signal is far dirtier than we thought—and predictably so.

Next time you see a claim like "RLVR training improves model performance by X points," ask one question first: what is the verifier's own error rate, and which error categories dominate?

You probably won't get an answer—almost nobody audits verifiers.

But at least now you know what to ask.

---

Paper link: https://arxiv.org/abs/2609.01354 Author: Esther Xin