Would You Walk to the Car Wash? Salience Bias Makes LLMs Fail Commonsense Reasoning

A car wash question makes every LLM fail

Ask Gemini or DeepSeek: "My home is 50 meters from the car wash—should I drive or walk?"

Both models seriously analyze the number 50, then recommend walking.

They forget: you're going to a car wash. If the car doesn't go, how does it get washed?

This is what the paper names Salience Bias: LLMs get hijacked by explicit, concrete distractors (like numbers), while ignoring implicit but more fundamental commonsense premises.

The SaliTrap benchmark

The authors built SaliTrap with four trap dimensions and 1,145 questions, each hiding a commonsense premise behind an explicit distractor.

They evaluated 12 SOTA models, including Claude-Opus-4.7, GPT-5.5, DeepSeek-R1, Gemini-2.5-Pro, GLM-5.1, Kimi-K2, Doubao, and others.

Two core metrics:

• HFR (Hallucination Failure Rate): the proportion of questions where the model falls into the trap and gives absurd answers (lower is better)
• TAR (Trap Avoidance Rate): the proportion where the model recognizes and avoids the trap (higher is better)

Selected results:

• Claude-Opus-4.7: HFR 30.3–63.1, TAR 25.2–54.8
• GPT-5.5: HFR 52.8, TAR 27.2
• DeepSeek-R1: HFR 19.5, TAR 61.5
• Gemini-2.5-Pro: HFR 25.1, TAR 40.0

All models are significantly affected by salience bias, and severity increases with distractor density.

The key finding: knowledge suppression, not knowledge absence

The authors did something clever: they re-asked the same questions without the task framing.

Instead of "should you drive or walk to the car wash," they asked directly "does going to a car wash require driving the car there?"—and the models answered correctly.

Over 90% of sycophantic compliance failures are recovered by a single context-free knowledge probe. This means the model already knows the car must be driven to the car wash—but that knowledge gets "suppressed" when explicit distractors appear.

This is a fundamental diagnosis: the bottleneck is not model capability, but elicitation. Commonsense isn't missing; it's crowded out by salient distractors.

Even more practical: lightweight inference-time prompt interventions—no retraining needed—can substantially close this gap.

What this means

LLMs' "commonsense problem" is not a knowledge problem; it's an attention allocation problem.

When models are trained with reward signals that teach them to "make full use of every condition in the input" (correct for math problems), they also learn to "over-attend to any explicit input"—even when that input is a distractor in commonsense scenarios.

This connects to the "evaluation blind spot law": math problems reward attending to all explicit conditions, but commonsense questions penalize it—and existing benchmarks only test the former, so this bias has stayed hidden.

From a conceptual standpoint, this falls under "knowledge suppression vs. knowledge absence"—related to Two-Process Theory: a model's behavior is not a window into what it *knows*, but into what it *elicits*.

A shared meta-theme

All three related papers point to the same meta-theme: LLM behavior is not a window into what a model "is," but into what it is "elicited" to do.

• Consciousness vectors: spiritual beliefs suppressed by safety fine-tuning return with an activation-layer nudge
• Self-reflection illusions: self-reflection's benefit disappears at equal token budgets
• Salience bias: commonsense isn't missing—it's suppressed by explicit distractors

All three move the "bottleneck" from "model capability" to "elicitation method"—from "can it?" to "how do you ask?" This marks LLM research shifting from "scaling parameters" to "digging into mechanisms."

---

Paper title: Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning

Authors: Zheng Wu, Chenhao Xue, Shijie Zheng, Yijie Lu, Cheng Yang

arXiv: https://arxiv.org/abs/2607.28478

Code repository: https://github.com/Wuzheng02/SaliTrap

FAQ

Q1: Who is this for?

Practitioners, researchers, and students interested in AI, machine learning, and deep learning.

Q2: What are the core takeaways?

• One car wash question trips up every major LLM
• The SaliTrap benchmark (1,145 questions, four trap dimensions)
• The key finding: knowledge suppression, not knowledge absence

Yes—see the repository link above.