User Feedback Is a Signal LLMs Can't Hear: LLM Judges Systematically Prefer Unimproved Answers

The Counterintuitive Finding

You ask an AI a question; it gives a flawed answer. You say, "The data in point three is wrong—it should be from 2023." The AI revises, and the answer is now correct.

Then you give both versions to another LLM acting as a judge and ask which is better.

It picks the uncorrected one.

This is not a joke—it is the core experimental result of the paper. Shachar Don-Yehiya et al. (Hebrew University + IBM Research) found a systematic phenomenon: when a model improves an answer because of user feedback, LLM judges systematically prefer the original, unimproved version. Across four different corruption types, LLM judges consistently rate the "unrepaired" answer above the "repaired" one—even though the repaired version objectively fixes more issues.

Experimental Design

The researchers built two datasets to isolate the real effect of feedback:

• Synthetic data: four types of corruption were deliberately injected into answers—factual errors, logical flaws, formatting issues, and relevance bias. The model then received user feedback and attempted repairs.
• Natural data: feedback and corrections collected from real conversations, validating whether the synthetic conclusions hold in the wild.

Key comparison: for the same question, the model generated corrected versions both with and without feedback, evaluated via issue resolution rate and LLM-judge preference.

Feedback Does Work

On issue resolution rate, the effect of feedback is clear:

• Synthetic data: with-feedback versions fixed more issues than without-feedback versions across all four corruption types.
• Natural data: the same pattern holds; the improvement from feedback is consistent.

Feedback is not noise. It carries information the model itself cannot generate—the user knows what the answer should look like, and that knowledge is not in the model's weights.

But Judges Can't Hear the Signal

The problem lies in evaluation. When LLM judges perform pairwise comparison:

• On samples with large improvements, judges consistently prefer the uncorrected version.
• The rate of preferring uncorrected versions far exceeds that of preferring corrected ones.
• Even when the correction objectively fixes the corruption, judges still pick the uncorrected answer.

Implication: if you use LLM-as-judge to evaluate feedback effectiveness, you will conclude "feedback doesn't help"—and you will be wrong.

Why Feedback Is a "Unique Signal"

The paper's key claim: the signal carried by user feedback is not encoded in model weights, so it cannot be obtained through distillation or self-improvement.

The logic is straightforward:

1. When generating an answer, the model can only rely on knowledge in its weights. 2. User feedback comes from external world knowledge—the user knows the correct answer; the model does not. 3. When the model revises based on feedback, the revised answer contains information the model itself could not produce. 4. The LLM judge is the same model (or a related model), equally lacking this external knowledge. 5. So the judge cannot recognize the value of the correction—it cannot see what it does not know.

This is isomorphic to "omission blindness": LLMs can detect what is present but not what should be present. Here, an LLM can evaluate answer quality but cannot evaluate whether an answer contains information it itself does not know.

A Challenge to the Evaluation Paradigm

The deeper contribution is exposing a systematic bias in the evaluation paradigm:

Current LLM evaluation assumes "the model knows what a high-quality answer looks like." But when answer quality depends on external information the model does not know, that assumption collapses.

This is not a tuning problem—it is structural: using model A to evaluate whether model A (or a related model B) benefits from external signals, model A can never see the parts it does not know.

The paper calls on the community to develop evaluation frameworks that reflect feedback's true utility. How exactly? The paper does not answer—and this open question may matter more than the finding itself.

Practical Implications

For teams doing RLHF and feedback-based training:

1. Do not rely solely on LLM-as-judge to evaluate feedback effectiveness. If your feedback signal genuinely carries information the model lacks, LLM judges may systematically underestimate it. 2. Build evaluation sets with ground truth. Assess feedback in settings with known correct answers rather than relying on model self-assessment. 3. Feedback's value is not in "making the model better" but in "making the model know what it doesn't know." If feedback only gets the model to repeat what it already knows, it truly is noise.

Conceptual Lineage

This paper resonates with several existing concepts:

• Omission blindness: LLMs can detect "what is" but not "what is missing." Feedback signals are precisely information about what should be there but isn't.
• Judgment–gate decoupling: a model may "know" feedback is useful (at the training-signal level), but the evaluation gate never consults that knowledge.
• Scalar illusion: managing "model quality" as a scalar ignores that "what the model knows" and "what the model doesn't know" are independent dimensions.

The finding that "user feedback is a unique signal" essentially says: a model's knowledge boundary is a real boundary, not a point on a scalar. Information beyond the boundary is invisible to the model—including to the model's own evaluators.

---

Paper: arXiv:2609.02859 Authors: Shachar Don-Yehiya, Leshem Choshen, Omri Abend, et al. (Hebrew University / IBM Research / MIT) Code: github.com/shachardon/feedback-blindspot Data: huggingface.co/datasets/shachardon/feedback-blindspot