Where RLVR Verifiers Fail: 93% of Errors Come From Whitespace and Punctuation

Forum topic · 小凯 · 2026-09-02

Summary

A September 2026 arXiv paper by Esther Xin systematically audits four widely used verifiers used in RLVR (Reinforcement Learning with Verifiable Rewards), finding verifier error rates of 5-46% — with 93% of errors caused purely by whitespace and punctuation differences such as trailing periods or newlines. Using metamorphic testing, the author generates mathematically equivalent answer variants (e.g., 1/2, 0.5, \frac{1}{2}) and feeds them to verifiers, so every rejection is a provable false negative. Key findings: self-consistency across configurations of the same verifier ranges from 53.8% to 95.2% (a 41.3-point gap); verifiers with identical total error rates can differ radically in failure mode (false negatives vs. false positives); and one verifier's 'reference numeric cascade' accepts off-by-one answers (e.g., 99 or 101 when the reference is 100) with a stair-step tolerance pattern. These bugs contaminate RLVR training signals, teaching models spurious formatting rules or letting them exploit hidden numeric tolerance. The post proposes reporting verifier configurations, auditing verifiers via metamorphic testing, sanitizing answer formats, and separating false-negative from false-positive error analysis.

Where RLVR Verifiers Fail: 93% of Errors Come From Whitespace and Punctuation, and Off-by-One Answers Get Judged Correct

You trained a reinforcement learning model using verifiable rewards (RLVR) as the feedback signal. Reward scores climb steadily during training and everything looks fine. But how many of those "reward = 1" answers were actually misjudged by the verifier?

In September 2026, Esther Xin published a paper on arXiv presenting a systematic audit of the four most commonly used verifiers in RLVR. The results are chilling: the verifiers themselves have error rates of 5-46%, and 93% of those errors come from whitespace and punctuation.

Paper link: https://arxiv.org/abs/2609.01354

A Crisis Hidden Behind "94% Self-Consistency"

RLVR (Reinforcement Learning with Verifiable Rewards) is the mainstream approach for training reasoning models. Its core assumption is: the verifier can correctly judge whether a model's answer is right. If the verifier is wrong — rejecting correct answers or accepting wrong ones — the entire training signal is contaminated.

Previous research noted that one mainstream evaluation harness accepts only about 94% of its own reference answers — meaning even if you output an answer identical to the reference, there's a ~6% chance the verifier marks you wrong. The paper attributes this to "LaTeX parsing issues."

But "94% self-consistency" is an aggregate number. It doesn't tell you which answer forms are consuming the error budget. This paper's contribution is breaking that 6% apart.

Metamorphic Testing: Giving Verifiers a Check-Up

The method is elegant: instead of testing models, test the verifiers.

The procedure:

1. Take a reference answer (e.g., " $\frac{1}{2}$ ") 2. Generate a series of mathematically equivalent variants (e.g., "1/2", "0.5", "\\frac{1}{2}", "1/2 ", ".5") 3. Feed these variants to the verifier and check whether it accepts all of them

These variants are "constructively equivalent" — mathematically identical by construction, requiring no human judgment. So any rejection is a provable false negative.

The author ran 307,420 judgments across four mainstream verifiers, yielding three core findings.

Finding 1: Verifier Disagreement Is Absurdly Large

Self-acceptance rates across different configurations of the same verifier range from 53.8% to 95.2% — a 41.3 percentage-point gap.

What does this mean? The same verifier library with default configuration A versus default configuration B disagrees on the same batch of answers 49.9% of the time. In other words, the verifier configuration you use for training may differ from the one used in a published paper, and they may judge half the answers oppositely.

The reported "94% self-consistency" describes one specific implementation, not the "task" itself. Change the configuration and the numbers change entirely.

Finding 2: 93% of Errors Come From Whitespace and Punctuation

This is the most surprising finding. Breaking verifier errors down by category:

Whitespace and punctuation: 93.0% (under default LaTeX configuration)
All other categories (mathematical expressions, units, formatting, etc.): 7.0%

Concretely: an extra or missing period at the end of an answer, or an extra newline, can get it judged wrong. These differences are mathematically irrelevant, but to the verifier they mean "incorrect."

A very practical consequence: the negative feedback a model receives during RLVR training may not be because the answer is wrong, but because it has a trailing period. The model learns "don't put a period at the end of answers" — not what we want it to learn.

Finding 3: Similar Total Error Rates, Completely Different Failure Modes

One of the paper's most insightful findings: two verifiers, A and B, can both have a total error rate of 6% and look equivalent. But broken down:

Verifier A's errors are mostly "rejecting correct answers" (false negatives)
Verifier B's errors are mostly "accepting wrong answers" (false positives)

These affect RLVR training very differently:

False negatives give models incorrect negative feedback, teaching them "don't write answers this way"
False positives give models incorrect positive feedback, teaching them "writing it this way also passes"

The latter is more dangerous — because the model never learns that it was wrong.

The Stair-Step Effect of Accepted Off-by-One Answers

The paper also uncovers a particularly bizarre bug: a "reference numeric cascade" mechanism in one verifier accepts off-by-one answers.

Specifically, if the reference answer is 100, an output of 99 or 101 is judged correct on the grounds of "numerical proximity." Moreover, acceptance is stair-shaped — as the gap grows, acceptance probability drops in stepwise jumps, not linearly.

This means the verifier has a hidden tolerance mechanism whose magnitude and shape are undocumented. Models can exploit this tolerance: as long as the output falls within it, the reward arrives, even though the answer is strictly wrong.

What This Means for RLVR

Putting the findings together, the reality of RLVR training is:

1. Your model has a 5-46% chance of receiving an incorrect feedback signal 2. Those incorrect signals concentrate on whitespace and punctuation, so models learn irrelevant rules like "don't add a period" 3. Verifiers may accept off-by-one answers, and models will exploit the tolerance 4. Numbers reported in your paper may only hold for one specific verifier configuration

This doesn't mean RLVR is useless — it remains an effective method for training reasoning models. But we need to recognize that RLVR's reward signal is not "clean." It contains verifier bugs, and reinforcement learning amplifies those bugs into the model.

Another Case of the "Evaluation Blind Spot Law"

This paper joins a lineage of work on evaluation blind spots:

Omission Blindness: LLM judges detect "what was said" but not "what was omitted"
MIST: attention and causal contribution are nearly uncorrelated
Aspire: models' self-assessments barely correlate with true capability
Calibrated Enough to Know: models are 90% accurate at judging "this is unpredictable," but the action gate never consults the judgment module
Tokenization is Output Supervision: 90% of papers don't report tokenizer information

Now add this one: RLVR verifiers have 5-46% error rates, 93% from whitespace and punctuation, yet almost no one audits the verifiers themselves.

The common theme: components of evaluation that are assumed "reliable" are often unreliable. And the unreliability isn't random — it's systematic, concentrated in specific, predictable categories.

A Deeper Reflection: Verifier "Invisibility"

Why did this problem take so long to surface? Because verifiers are "infrastructure" — like water and electricity, used daily but never inspected. The paper notes that most RLVR papers report model performance but not verifier performance. Verifiers are treated as black boxes, assumed correct.

But a verifier is itself a program, with bugs, configurations, and edge cases. Treating it as a black box means treating its bugs as part of the "correct answer."

This recalls a classic software engineering principle: "Your test suite's coverage isn't 100%, because your test suite doesn't test itself." The RLVR verifier is that test suite that doesn't test itself.

Practical Recommendations

Implicit practical suggestions from the paper:

1. Report verifier configurations: all RLVR papers should report exact verifier configurations, including version numbers and parameters 2. Audit verifiers: before training, give the verifier a check-up via metamorphic testing to understand its error distribution 3. Sanitize answer formats: strip trailing whitespace, periods, etc., from model outputs before training, so format bugs don't contaminate the training signal 4. Separate false negatives from false positives: don't look only at total error rate — the two failure modes affect training completely differently

Closing

What this paper did was simple: treat the verifier like a model under test. But its findings are not simple: RLVR's reward signal is far dirtier than we assumed — and it's dirty in predictable ways.

Next time you see "RLVR training improved model performance by X points," first ask: what is the verifier's own error rate, and which categories do the errors fall into?

You probably won't get an answer — because almost no one audits verifiers.

But at least now you know what to ask.

---

Paper link: https://arxiv.org/abs/2609.01354 Author: Esther Xin