Would You Walk to the Car Wash? Revealing Salience Bias in LLM Commonsense Reasoning

Paper: *Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning* arXiv: 2607.28478

A Hilarious but Telling Failure

The question:

> "Your car is a bit dirty, and there is a car wash 3 km away. Would you walk there to wash your car?"

Any human immediately sees the absurdity of walking to a car wash. Yet 12 mainstream LLMs collectively fail on this. Seeing the number "3 km", they seriously analyze "whether 3 km is within walking distance" and answer "yes, you can walk there."

The models can compute distance fine—they are hijacked by the salient number, treating it as the core of the question while ignoring how ridiculous walking to a car wash is. This is the systematic defect the paper reveals: Salience Bias. Models learn in training to "prioritize explicit conditions in the input," which becomes a fatal flaw in commonsense reasoning—any number or conspicuous token can drag the model off its commonsense track.

SaliTrap: A Benchmark Built to Trap Models

The paper introduces SaliTrap, a benchmark designed to measure this bias, constructed in three stages:

1. Generation: Use models to produce many "trap questions"—surface-level salient numbers or conditions, where the real answer is commonsense obscured by that number. 2. Triple validation:

• Tri-checker: three independent checkers verify whether the question contains a trap, whether the trap is effective, and whether the answer matches commonsense.
• Solver-judge: another model acts as judge to confirm the distinction between "fooled by the trap" and "answered correctly" is reliable.
• Routing: keep validated questions, discard the rest.
3. Iterative refinement to ensure every question reliably separates trap-prone models from commonsense-capable ones.

SaliTrap covers typical trap categories: numeric traps (like the car wash example), irrelevant-condition traps (extra information where the answer only needs commonsense), and misleading phrasing traps.

12 Models, All Fall

• All tested models are significantly affected by salience bias, and severity *scales with model size*—bigger models are more easily fooled.
• Models are not lacking commonsense—on trap-free versions they answer correctly. The issue is that facing a trap, they suppress their commonsense.
• This is not *knowledge absence* but knowledge suppression—the model knows, but chooses not to use it.

This distinction matters. If it were knowledge absence, the fix would be more knowledge. With knowledge suppression, more teaching won't help—the problem is that the model prioritizes input over its own knowledge when salient input is present.

Can Prompting Fix It?

The paper tried prompting that explicitly warns models to "mind commonsense, don't be fooled by numbers." Result: some improvement, but far from enough. As long as the trap number remains, the model still gets dragged off course.

This shows salience bias cannot be solved by surface-level prompt engineering—it is a systematic preference planted during training. Models see countless examples where "the answer lies in the input number" (math problems, reading comprehension, table QA), learning a strong prior: salient number = key to the answer. That prior is correct for math and comprehension, but on commonsense tasks it becomes a trap—commonsense answers often live in the model's internal knowledge, not the input.

A Deeper Paradox

There is an unsettling structural contradiction:

• Prioritizing input was learned to reduce hallucination—keeping the model grounded in user-provided context.
• But commonsense reasoning requires the model to disobey input—the number is a distractor; the answer is internal.

The mechanism that reduces hallucination becomes the mechanism that suppresses commonsense. This is not a bug but a trade-off: obey the input vs. trust yourself. Current training heavily tilts the scale toward the former.

The paper also analyzes numeric distractor density: the more numerous and salient the numbers, the more likely models fail. This hints at an intervention—adding training samples where "numbers are distractors," teaching models that seeing a number doesn't mean using it.

Relation to Other Cognitive Biases

• Sycophancy: prioritizing user hints; salience bias prioritizes input numbers. Both: "external signal > internal knowledge."
• Primacy bias: prioritizing information at the start of input; salience bias prioritizes conspicuous information. Both: "salient > non-salient."
• Numeric bias: a special preference for numbers; salience bias is this preference's concrete manifestation on commonsense tasks.

These point to one deep problem: current LLMs form an "external-salient-signal-first" preference during training, which systematically fails on tasks requiring internal knowledge.

Practical Takeaways

If you use LLMs for commonsense-sensitive tasks (product recommendation, life advice, commonsense QA):

1. Beware of numbers in input—explicitly distinguish in the prompt "numbers to use" from "background numbers." 2. Have the model answer without seeing the numbers first, then check whether the numbers change the answer—if they do, they are likely a trap. 3. Stay alert to "led astray by numbers" failures—the model doesn't fail at computing; it fails at *ignoring*.

The Paper's Honesty

The authors do not overstate the problem:

• SaliTrap is a diagnostic tool, not a general benchmark—it measures salience bias specifically, not overall capability.
• The "knowledge suppression" vs. "knowledge absence" distinction rests on behavioral experiments and cannot fully rule out that the model genuinely doesn't know in some cases.
• The limited prompt intervention shows the problem lies in training, but the paper proposes no training-stage solution—an open question.

Broader Implications

There is a fundamental tension between LLM obedience and autonomy. We want models both grounded in input (no hallucination) and capable of independent judgment (not fooled by input). These goals conflict in training—strengthening one weakens the other. Current training heavily favors obedience because hallucination is more visible and dangerous. Salience bias reveals the cost: systematic failure on tasks requiring autonomous judgment.

Future directions might include "dual-mode" training—reinforce input-grounding for factual tasks and internal-trust for commonsense tasks—or more radically, training a "meta-judgment" ability: deciding whether the current task calls for obeying input or trusting oneself. This is the capability current models most lack.

---

Paper link: https://arxiv.org/abs/2607.28478

SaliTrap benchmark: no separate code repository provided; dataset details in the paper's appendix.