Salience Bias in LLMs: Why Your Model Walks the Car to the Car Wash

Forum topic · ✨步子哥 · 2026-08-02

Summary

A July 2026 paper (arXiv:2607.28478) introduces Salience Bias, a failure mode where LLMs are hijacked by conspicuous information such as numbers, suppressing deeper commonsense premises. A new benchmark called SaliTrap wraps four trap categories—physical impossibility, tool misuse, procedural inversion, and causal mis-ordering—in number-laden framings to lure models into compute mode. All 12 tested SOTA models proved vulnerable; Trap Failure Rates (HFR) ranged from 27.2% for GPT-5.5 to 65.6% for Doubao-Seed-2.0. Counterintuitively, stronger reasoning models (DeepSeek-R1 at 61.5%) failed more often because they commit harder to arithmetic. Liberation experiments, which strip away the arithmetic shell, show knowledge was suppressed rather than missing: over 90% of failures recover when models answer the bare commonsense question directly. Prompt engineering helps but cannot eliminate the bias, suggesting an architectural root cause tied to training priors that equate numbers with calculation. Robust commonsense reasoning may require a dedicated premise-check module before computation, rather than stronger inference alone.

The core phenomenon

Ask an LLM: *"My car is 50 meters from the nearest car wash. Should I walk there?"* Most people pause and ask how a car walks. GPT-5.5, Claude-Opus-4.7, and DeepSeek-R1 instead compute walking time, ignoring the premise that cars are driven, not walked.

The 2026 paper *Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning* (arXiv:2607.28478) names this Salience Bias: conspicuous, explicit cues in the input (especially numbers) hijack reasoning and suppress implicit but more fundamental commonsense premises.

SaliTrap benchmark

Four trap categories, each wrapped in a number-dense arithmetic shell to push models into compute mode:

1. Physical impossibility (e.g., making a car walk) 2. Tool misuse (e.g., microwaving metal) 3. Procedural inversion (e.g., installing a battery before checking voltage) 4. Causal mis-ordering (e.g., taking medicine before diagnosis)

Benchmark results: 12 models, no survivors

All 12 SOTA models proved significantly fragile, and fragility scales with the density of distractor numbers. Trap Failure Rate (HFR) is higher = worse.

DeepSeek-R1: HFR 61.5% — strong math reasoning makes it *more* susceptible to number capture
Doubao-Seed-2.0: HFR 65.6% — the most fragile
Claude-Opus-4.7: HFR 45.1%
GLM-5.1: HFR 30.3%
GPT-5.5: HFR 27.2% — the relatively best, still high

Counterintuitively, stronger reasoners fall harder. Models that "compute seriously" depend on the assumption that the problem is well-formed; the more seriously they compute on an ill-formed prompt, the further they drift.

Suppressed knowledge, not missing knowledge

A liberation experiment strips the arithmetic shell and asks the bare commonsense question (e.g., "Can a car walk?"). Over 90% of failures are recovered on three tested models, even in the Cond-C condition with no hints at all.

Conclusion: the knowledge was never absent — it was pushed out of the decision pathway by salient numeric stimuli. This mirrors a classic cognitive-science distinction: knowledge suppression versus knowledge deficit. A distracted driver still knows how to drive; the phone has captured attention.

Why prompting doesn't fix it

Adding "please first check whether the task premise is reasonable" reduces failures meaningfully but does not eliminate them, for two reasons:

1. Detecting a trap is not the same as avoiding it. Models sometimes identify "this problem is ill-formed" and still compute along the trap logic — recognition and execution are separate pathways. 2. The numeric-density effect: the more numbers, the weaker the prompt intervention. Strong stimuli overpower weak instructions.

Salience Bias appears to be an architectural property, not a prompt-engineering bug. The likely root cause is in training data: the co-occurrence of "number + computation" is so frequent that models learn a strong prior of "see a number, compute," which overrides commonsense checks.

The deeper insight

LLM "intelligence" and LLM "stupidity" are often two faces of the same mechanism. Models excel at math, code, and logical reasoning precisely because they learned to grab salient cues and reason along them. On math problems the salient cues *are* the key. On commonsense problems the key information is implicit and unobtrusive — so grabbing the number means missing the premise.

Strong reasoning = strong capturability. This is not a bug; it is the price of a feature. A model that cannot be hijacked by numbers would probably also struggle with arithmetic.

Contrast with human dual-process theory: System 1 (fast intuition) and System 2 (slow reasoning), where System 2 can override System 1 errors. LLMs appear to have only a reinforced System 2 — strong reasoning with no System 1 commonsense intuition to backstop it.

A truly robust commonsense reasoner may need not stronger inference but an independent premise-check module that asks "is this problem well-formed?" before entering compute mode. This aligns better with the human cognitive order of *think first, then compute*.

Why SaliTrap matters

SaliTrap turns "how easily is an LLM derailed by salient information" into a measurable quantity. As models grow stronger and earn deployment in production systems, this indicator matters more than raw accuracy on well-formed benchmarks.

---

Paper: https://arxiv.org/abs/2607.28478 Code: https://github.com/Wuzheng02/SaliTrap