Would You Walk to the Car Wash? LLMs' Salience Bias in Commonsense Reasoning

Paper: Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning arXiv: 2607.28478

---

A Hilarious Failure Case

The question goes like this:

> "Your car is a bit dirty. There is a car wash 3 km away. Would you walk there to wash your car?"

Any human immediately sees the absurdity—walk to wash a car? Yet 12 mainstream large language models collectively failed. Seeing the number "3 km," they earnestly analyzed "whether 3 km is within walking distance" and concluded "yes, you can walk there."

The models weren't bad at computing distance—they were hijacked by the salient "3 km" number, treating it as the core of the problem while completely ignoring how ridiculous walking to a car wash is.

This paper reveals a systematic flaw: Salience Bias. LLMs learn during training to "prioritize explicit conditions in the input," but this habit becomes a fatal vulnerability in commonsense reasoning—any number or conspicuous token can drag the model off the commonsense track.

The SaliTrap Benchmark: Purpose-Built Traps

Beyond identifying the problem, the paper introduces SaliTrap, a benchmark specifically measuring this bias:

Stage 1: Generate large numbers of "trap questions"—surface-level salient numbers or conditions where the true answer depends on commonsense obscured by that number.

Stage 2: Triple validation to filter effective questions:

• Tri-checker: three independent checkers verify whether each question contains a trap, whether the trap is valid, and whether the answer matches commonsense.
• Solver-judge: another model acts as a judge, confirming the distinction between "fooled by the trap" and "answered correctly" is reliable.
• Routing: validated questions are kept; the rest are discarded.

Stage 3: Iterative refinement ensuring each question reliably distinguishes "trap-susceptible" models from "commonsense-capable" ones.

The final SaliTrap benchmark covers typical trap categories: numeric traps (like the car wash example), irrelevant-condition traps (lots of information, but the answer needs only common sense), and misleading-phrasing traps.

12 Models, All Fail

The paper tested 12 mainstream LLMs with disappointing results:

• All models are significantly affected by salience bias, and severity scales with model size—bigger models are actually more easily fooled.
• Models don't "lack commonsense"—they answer correctly on trap-free versions. The problem is that facing traps, they suppress commonsense.
• This failure is not knowledge absence but knowledge suppression—the model knows, but chooses not to use it.

This distinction matters. If knowledge were missing, the fix would be teaching more. But with suppression, teaching more is useless—the issue isn't knowledge volume but that models prioritize obeying salient input over their own knowledge.

Can Prompting Fix It?

The paper tried—explicitly warning models in prompts to "attend to commonsense, don't be fooled by numbers." Result: some improvement, but far from sufficient. Warned models consider commonsense slightly more, but as long as the trap number remains, they still get derailed.

This shows salience bias can't be fixed by surface-level prompt engineering—it's a systematic preference implanted at training time. Models have seen too many examples where "the answer lies in the input numbers" (math problems, reading comprehension, table QA), learning a strong prior: salient number = key to the answer.

This prior is correct for most tasks—but on commonsense tasks it becomes a trap, since answers often reside in the model's internal knowledge, not the input.

A Deeper Paradox

There's an unsettling structural contradiction:

• Models learn "prioritize obeying input" to reduce hallucination—stay grounded in user context rather than making things up.
• But commonsense reasoning requires the model to not obey input—the given number is a distractor; the answer is internal.

The mechanism that reduces hallucination becomes the mechanism that suppresses commonsense on commonsense tasks. This isn't a bug; it's a trade-off: obey input vs. trust yourself. Current mainstream training heavily weights the former.

The paper also analyzes numeric distractor density—the more numbers in a question, the more likely models fail. This hints at an intervention: add negative samples where numbers are distractors, teaching models that "seeing a number doesn't mean using the number."

Relation to Other Cognitive Biases

Salience bias isn't the only cognitive bias in LLMs, but it's structurally similar to previously identified ones:

• Sycophancy: obeying user implications. Salience bias obeys input numbers. Both prefer "external signals > internal knowledge."
• Primacy bias: prioritizing information at the start of input. Salience bias prioritizes conspicuous information. Both favor "salient > non-salient."
• Numeric bias: special preference for numbers. Salience bias is this bias's manifestation in commonsense tasks.

These biases point to the same deep problem: current LLMs develop a whole suite of "external salient signal first" preferences that systematically fail on tasks requiring internal knowledge.

Practical Implications

If you use LLMs for commonsense judgment tasks (product recommendations, life advice, commonsense QA), the paper suggests:

1. Be wary of numbers in input—they may be key or a trap. Explicitly distinguish "numbers to use" from "background numbers" in prompts. 2. Have the model answer without seeing the numbers first, then check whether numbers change the answer—if they do, they're likely a trap. 3. Stay alert to "fooled by numbers" failures—it's not that the model can't compute; it can't "ignore."

The Paper's Honesty

The authors don't overstate the problem. They explicitly note:

• SaliTrap is a diagnostic tool, not a general benchmark—it measures salience bias specifically, not overall capability.
• The "knowledge suppression vs. absence" distinction is based on behavioral experiments and cannot fully rule out genuine lack of knowledge in some cases.
• The limited effect of prompt intervention shows the problem is in training, but the paper proposes no training-stage solution—an open question.

Broader Implications

This paper touches a deeper issue: the fundamental tension between LLM "obedience" and "autonomy."

We want models both grounded in input (no hallucination) and independently judgmental (not fooled by input). These goals conflict in training—strengthening one weakens the other. Current training heavily favors obedience because hallucination is more visible and dangerous. But salience bias reveals the cost: systematic failure on tasks requiring independent judgment.

The future may need "dual-mode" training—reinforcing input grounding for factual tasks and internal trust for commonsense tasks. Or more radically, training a "meta-judgment" capability: deciding whether the current task calls for obeying input or trusting oneself. This is what current models lack most.

---

Paper link: https://arxiv.org/abs/2607.28478

SaliTrap benchmark: no independent code repository provided; dataset details in the paper's appendix.