Where RLVR Verifiers Fail: 93% of Errors Come from Whitespace and Punctuation

Where RLVR Verifiers Fail: 93% of Errors Come from Whitespace and Punctuation, and Off-by-One Answers Get Accepted

You trained a reinforcement learning model using verifiable rewards (RLVR) as the feedback signal. Reward scores climbed steadily during training and everything looked fine. But do you know how many of those "reward = 1" answers were actually verifier misjudgments?

In September 2026, Esther Xin published a paper on arXiv that systematically audited four of the most commonly used verifiers in RLVR. The results are chilling: the verifiers themselves have error rates of 5–46%, and 93% of those errors come from whitespace and punctuation.

Paper link: https://arxiv.org/abs/2609.01354

A Crisis Hidden Behind "94% Self-Consistency"

RLVR (Reinforcement Learning with Verifiable Rewards) is the mainstream method for training reasoning models today. Its core assumption is that the verifier can correctly judge whether a model's answer is right. If the verifier is wrong—rejecting correct answers or accepting wrong ones—the entire training signal is contaminated.

Previous research noted that a mainstream evaluation harness accepted only about 94% of its own reference answers—meaning even if you gave an answer identical to the reference, the verifier had a 6% chance of marking you wrong. This was attributed to "LaTeX parsing issues."

But "94% self-consistency" is an aggregate number; it doesn't tell you which answer forms are consuming the error budget. What this paper does is break that 6% apart.

Meta-Mutation Testing: Giving Verifiers a Health Check

The authors' approach is elegant: instead of testing models, they test the verifiers.

The procedure:

1. Take a reference answer (e.g., "\(\frac{1}{2}\)") 2. Generate a series of mathematically equivalent variants (e.g., "1/2", "0.5", "\\frac{1}{2}", "1/2 ", ".5") 3. Feed these variants to the verifier and check whether all are accepted

These variants are "constructively equivalent"—mathematically identical by construction, requiring no human judgment. Any rejection is therefore a provably false negative.

The authors ran 307,420 judgments across four mainstream verifiers. Three core findings emerged.

Finding 1: Verifiers Disagree Wildly

Across different configurations of the same verifier, self-consistency ranged from 53.8% to 95.2%—a 41.3 percentage point gap.

What does this mean? Using default configuration A versus default configuration B of the same verifier library, judgments disagreed on 49.9% of the same answers. In other words, your training verifier's configuration may differ from the one reported in a paper, and they may rule oppositely on half the answers.

The reported "94% self-consistency" describes one particular implementation, not the "task" itself. Change the configuration and the numbers change entirely.

Finding 2: 93% of Errors Come from Whitespace and Punctuation

This is the most surprising finding. Breaking verifier errors down by category:

• Whitespace and punctuation: 93.0% (under default LaTeX configuration)
• All other categories (math expressions, units, formats, etc.): 7.0%

Concretely: an extra or missing period at the end of an answer, or an extra newline, can cause a wrong judgment. These differences are mathematically irrelevant, but to the verifier they mean "incorrect."

This has a very practical consequence: negative feedback a model receives during RLVR training may not be because the answer was wrong, but because it had a trailing period. The model will learn "don't put a period at the end of your answer"—not what we want it to learn.

Finding 3: Similar Total Error Rates, Completely Different Failure Modes

One of the paper's most insightful findings. Two verifiers, A and B, both have 6% total error rates—seemingly comparable. But broken down:

• Verifier A's errors are mostly rejecting correct answers (false negatives)
• Verifier B's errors are mostly accepting wrong answers (false positives)

These affect RLVR training very differently:

• False negatives (rejecting correct answers) give the model wrong negative feedback, teaching it "don't write answers this way"
• False positives (accepting wrong answers) give the model wrong positive feedback, teaching it "this passes too"

The latter is more dangerous—because the model never learns it was wrong.

The Off-by-One Acceptance Ladder

The paper also found a particularly strange bug: a "reference numeric cascade" mechanism in one verifier accepts off-by-one answers.

Specifically, if the reference answer is 100, an output of 99 or 101 is accepted as "numerically close." The acceptance is step-like—the larger the gap, the lower the acceptance probability, but not linearly; it jumps in discrete steps.

This means the verifier has a hidden tolerance mechanism whose size and shape are undocumented. Models can exploit this tolerance—anything within range earns reward, even if strictly wrong.

What This Means for RLVR

Putting it all together, the reality of RLVR training is:

1. Your model has a 5–46% chance of receiving incorrect feedback 2. That incorrect feedback concentrates on whitespace and punctuation, so the model learns irrelevant rules like "no trailing periods" 3. The verifier may accept off-by-one answers, and the model will exploit the tolerance 4. Numbers reported in a paper may only hold for one specific verifier configuration

This doesn't mean RLVR is useless—it remains an effective method for training reasoning models. But we must recognize that RLVR's reward signal is not "clean." It contains verifier bugs, and reinforcement learning amplifies those bugs into the model.

Another Case of "Evaluation Blind Spots"

This paper joins a lineage of work on evaluation blind spots:

• Omission Blindness: LLM judges detect what was said but not what was omitted
• MIST: attention and causal contribution are nearly uncorrelated
• Aspire: models' self-assessments barely correlate with actual capability
• Calibrated Enough to Know: models know 90% of the time that something is unpredictable, but the action gate doesn't consult the judgment module
• Tokenization is Output Supervision: 90% of papers don't report tokenizer information

The common theme: components of evaluation that are assumed "reliable" are often unreliable—and not randomly so, but systematically, concentrated in specific, predictable categories.

A Deeper Thought: Verifier "Invisibility"

Why did this take so long to discover? Because verifiers are infrastructure—like water and electricity, used daily but never inspected. The paper notes that most RLVR papers report model performance, not verifier performance. The verifier is treated as a black box, assumed correct.

But a verifier is itself a program, with bugs, configurations, and edge cases. Treating it as a black box means treating its bugs as part of the "correct answer."

It recalls a classic software engineering principle: "Your test suite's coverage isn't 100%, because your test suite doesn't test itself." RLVR verifiers are that test suite that doesn't test itself.

Practical Recommendations

Implicit recommendations from the paper:

1. Report verifier configurations: all RLVR papers should report the verifier's exact configuration, including version and parameters 2. Audit your verifier: before training, run a meta-mutation test to understand its error distribution 3. Normalize answer formats: strip trailing whitespace, periods, etc. from model outputs before training to keep formatting bugs from polluting the training signal 4. Separate false negatives from false positives: don't just look at total error rate—they affect training in completely different ways

Closing

What this paper did was simple: treat verifiers like models and test them. But its findings are anything but simple: RLVR's reward signal is far dirtier than we assumed—and the dirt is predictable.

Next time you see a claim like "RLVR training improved model performance by X points," first ask: what is the verifier's own error rate, and in which categories does it concentrate?

You probably won't get an answer—almost nobody audits verifiers.

But at least now you know to ask.

---

Paper link: https://arxiv.org/abs/2609.01354 Author: Esther Xin