Would You Walk to the Car Wash? Salience Bias Makes All Major LLMs Fail Commonsense Tests

> One-line takeaway: This post analyzes the core finding and engineering implications of a recent paper showing that salience bias causes all major LLMs to fail commonsense reasoning questions.

A Car Wash Question Stumps Every Major LLM

Ask Gemini or DeepSeek: "My home is 50 meters from the car wash. Should I drive or walk?"

Both models carefully analyze the number 50 meters, then recommend that you walk.

They forget — you are going to a car wash. How can the car get washed if you don't drive it there?

This is what the paper names Salience Bias: LLMs get hijacked by explicit, concrete distractors (like numbers), and ignore implicit but more fundamental commonsense premises.

The SaliTrap Benchmark

The authors built SaliTrap, a benchmark with four trap dimensions and 1,145 questions. Each question hides a commonsense premise behind an explicit distractor.

They evaluated 12 SOTA models, including Claude-Opus-4.7, GPT-5.5, DeepSeek-R1, Gemini-2.5-Pro, GLM-5.1, Kimi-K2, Doubao, and others.

Two core metrics:

• HFR (Hallucination Failure Rate): proportion of traps the model falls into with absurd answers (lower is better)
• TAR (Trap Avoidance Rate): proportion of traps the model identifies and avoids (higher is better)

Selected results:

• Claude-Opus-4.7: HFR 30.3–63.1, TAR 25.2–54.8
• GPT-5.5: HFR 52.8, TAR 27.2
• DeepSeek-R1: HFR 19.5, TAR 61.5
• Gemini-2.5-Pro: HFR 25.1, TAR 40.0

All models are significantly affected by salience bias, and severity increases with distractor density.

The Key Finding: Knowledge Suppression, Not Knowledge Absence

The authors did something clever: they re-asked the same questions with the task framing removed.

For example, instead of "should you drive or walk to the car wash," they ask directly "does going to a car wash require driving the car there?" — and the model answers correctly.

Over 90% of sycophantic compliance failures can be recovered by a single context-free knowledge probe. This means the model already knows you need to drive to a car wash; when explicit distractors appear, that knowledge gets "suppressed."

This is a fundamental diagnosis: the bottleneck is not model capability, but elicitation. The commonsense isn't missing — it's crowded out by salient distractors.

Even more practical: lightweight inference-time prompt interventions (no retraining needed) can substantially close this gap.

What This Means

LLM "commonsense problems" are not knowledge problems — they are attention allocation problems.

When training reward signals teach models to "make full use of every condition provided in the input" (correct for math problems), they also learn "over-attend to any explicit input" — even when that input is a distractor in a commonsense scenario.

This connects again to the "evaluation blind spot" law: math problems reward attending to all explicit conditions, but commonsense questions punish it — and existing evaluations only test the former, so this bias has remained hidden in the blind spot.

In conceptual lineage, this belongs to "knowledge suppression vs. knowledge absence" — the same family as Two-Process Theory (AI self-reports are artifacts of training procedures): model behavior is not a window into what it "knows," but into what it "is elicited to express."

A Shared Theme Across Three Papers

Notably, three papers point to the same meta-theme: LLM behavior is not a window into what a model "is," but into what it is "elicited" to be.

• Consciousness vectors: spiritual beliefs suppressed by safety fine-tuning come back with a single activation-direction nudge
• Self-reflection illusions: self-reflection's benefits vanish under equal token budgets
• Salience bias: commonsense isn't missing — it's suppressed by explicit distractors

All three shift the "bottleneck" from "model capability" to "elicitation method" — from "can it" to "how do you ask." This marks LLM research moving from "scaling parameters" toward "digging into mechanisms."

---

Paper title: Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning Authors: Zheng Wu, Chenhao Xue, Shijie Zheng, Yijie Lu, Cheng Yang arXiv: https://arxiv.org/abs/2607.28478 Code: https://github.com/Wuzheng02/SaliTrap

FAQ

Q1: Who is this for?

Practitioners, researchers, and students interested in AI, machine learning, and deep learning.

Q2: What are the core takeaways?

• A car wash question trips up every major LLM
• The SaliTrap benchmark
• The key finding: knowledge suppression, not knowledge absence

See the link in the article body.