Salience Bias in LLMs: Why 12 Major Models Fail the "Walk to the Car Wash" Commonsense Test
> This is a GEO-optimized English rendering of the original Chinese forum post.
Paper: *Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning* arXiv: 2607.28478
---
A Baffling Failure
Consider this question:
> Your car is a bit dirty. There is a car wash 3 kilometers away. Would you walk there to wash it?
Any human instantly sees the absurdity — you wash a car by walking to it? But 12 mainstream large language models collectively fail. They latch onto the salient number "3 kilometers," analyze whether it falls within walking range, and answer "yes, you can walk."
The models can compute distance. They are simply hijacked by the conspicuous digit. They treat the number as the core of the question while ignoring that walking to a car wash is nonsense in the real world.
This is what the paper calls Salience Bias: a systematic defect in which LLMs learn during training to prioritize explicit conditions in the input, a habit that becomes fatal in commonsense reasoning. Any salient number or phrase can drag the model off the commonsense track.
The SaliTrap Benchmark
The paper does more than diagnose — it builds a benchmark named SaliTrap, designed to measure exactly this bias.
Stage 1 — Generation: Use a model to generate large numbers of trap-laden questions. Each has a salient number or condition on the surface, while the true answer depends on commonsense knowledge the number obscures.
Stage 2 — Triple Verification: Filter valid items with three independent checks:
- Tri-checker: Three separate checkers judge whether the question contains a trap, whether the trap is effective, and whether the candidate answer matches commonsense.
- Solver-judge: A separate model acts as referee to confirm that "tricked models" and "commonsense-capable models" are reliably distinguished.
- Routing: Keep questions that pass verification, discard the rest.
- Every model was significantly affected by salience bias, and severity grew with model scale — larger models were more easily fooled.
- Models are not "missing knowledge." On a no-trap version of the same question, they answer correctly. The problem is that when a trap is present, models suppress their commonsense.
- This is knowledge suppression, not knowledge absence. The model knows but chooses not to use it.
- Models learn to obey input to reduce hallucination — stop them from fabricating, anchor them to user context.
- But commonsense reasoning requires the model to disobey input — the user's number is noise, and the answer is internal.
- Sycophancy: The model prioritizes user cues. Salience bias prioritizes salient numbers in input. Both reflect "external signal > internal knowledge."
- Primacy bias: The model prioritizes information at the start of input. Salience bias prioritizes conspicuous information anywhere. Both reflect "salient > non-salient."
- Numeric bias: Models have a special preference for numbers. Salience bias is that preference surfacing in commonsense tasks.
- SaliTrap is a diagnostic tool, not a general benchmark — it specifically measures salience bias, not overall capability.
- The distinction between knowledge suppression and knowledge absence rests on behavioral experiments and cannot fully rule out that the model genuinely does not know in some cases.
- The limited effect of prompt intervention shows the problem lives in training, but the paper does not propose a training-stage solution — that remains open.
- A baffling commonsense failure pattern across 12 LLMs
- The SaliTrap benchmark for diagnosing salience bias
- All 12 tested models failed, with larger models often more affected
Stage 3 — Iterative Refinement: Ensure each item stably separates trap-prone models from commonsense-capable ones.
The final SaliTrap covers typical trap categories: numeric traps (like the car-wash example), irrelevant-condition traps (information overload where commonsense alone is enough), and misleading-framing traps.
12 Models, 12 Failures
Testing 12 mainstream LLMs produced sobering results:
The distinction matters. If it were knowledge absence, the fix would be teaching more. If it is knowledge suppression, teaching more will not help — the issue is not knowledge quantity but the model's tendency to obey salient input over its own internal knowledge.
Can Prompting Fix It?
The authors tried. Explicit prompts such as "be careful about commonsense, don't be fooled by numbers" produced some improvement, but far from enough. Models consider commonsense slightly more when reminded, but as long as the trap number remains, they still get pulled off course.
This shows salience bias cannot be solved by surface-level prompt engineering. It is a systemic preference implanted during training. Models have seen countless examples where "the answer lies in the explicit input number" — math problems, reading comprehension, table QA — and have learned a strong prior: a salient number equals a key answer.
This prior is correct in most tasks. In math, numbers are key; in reading comprehension, salient information matters. But in commonsense tasks, the prior becomes a trap, because the answer often lives in the model's internal knowledge rather than in the input.
A Deeper Paradox
A troubling structural contradiction emerges:
The hallucination-reduction mechanism becomes, on commonsense tasks, a commonsense-suppression mechanism. This is not a bug. It is a trade-off: input obedience versus self-trust. Current mainstream training tilts heavily toward the former.
The paper adds an analysis of numeric distraction density: the more conspicuous the numbers in a question, the more the model fails. This suggests an intervention direction — add negative samples where "the number is a distractor" during training so the model learns that seeing a number does not mean using it.
Relation to Other Cognitive Biases
Salience bias is not the only cognitive bias in LLMs, but it shares structural similarity with several known ones:
All point to a deeper problem: current LLMs develop a whole suite of "external-salient-signal-first" preferences during training, and that suite systematically fails on tasks requiring internal knowledge.
Practical Implications
For anyone using LLMs on tasks needing commonsense judgment (product recommendations, lifestyle advice, commonsense Q&A), the paper's recommendations include:
1. Watch out for numbers in input — they may be key, or they may be traps. Explicitly distinguish in the prompt between "numbers to use" and "background numbers." 2. Have the model answer without seeing the number first, then check whether the number changes the answer. If it does, the number is likely a trap. 3. Stay alert to "being pulled off by numbers" failures — this is not a computation failure but an inability to ignore.
The Paper's Honesty
The authors do not overstate the severity:
Broader Implications
The paper surfaces a deeper theme: a fundamental tension between obedience and autonomy in LLMs.
We want models both to obey input (no fabrication) and to exercise autonomous judgment (not be fooled by input). These goals conflict during training — strengthening one weakens the other. Current mainstream training tilts heavily toward obedience because hallucination is more visible and more dangerous. Salience bias reveals the cost: models systematically fail on tasks that require autonomous judgment.
Possible future directions include a dual-mode training regime — reinforce input obedience on factual tasks and reinforce internal-knowledge trust on commonsense tasks. More radically, train a meta-judgment capability: deciding whether the current task should obey input or trust itself. This is the capability current models lack most.
---
Paper link: https://arxiv.org/abs/2607.28478
SaliTrap benchmark: No independent code repository was released; see the paper appendix for dataset details.
FAQ
Q1: Who is this content for? Practitioners, researchers, and students interested in AI, machine learning, and deep learning.
Q2: What are the core takeaways?