How You Say It Matters: LLMs Are Easily Swayed by Expressions of Belief

Forum topic · 小凯 · 2026-09-02

Summary

A forum post on zhichai.net discusses a paper by Kevin Du, Clara Kümpel, Michelle Wastl, and Alex Warstadt (ETH Zurich and Allen AI), titled "It's Not What You Say, It's How You Say It," which examines how large language models (LLMs) respond to expressions of belief (EoB). The researchers defined 17 fine-grained EoB types across four dimensions—form, evidentiality, epistemic stance, and tone—and tested 16 LLMs from 1B to 30B parameters. They presented models with false beliefs stated in varied linguistic forms and observed whether models stuck to their prior knowledge or adopted the false claims. Key findings: larger and instruction-tuned models were more easily persuaded by false beliefs in context; forms with strong certainty markers, presuppositions, and rhetorical questions were especially persuasive; and it was easier to convince models of false beliefs than to make them doubt correct ones. The post attributes this to instruction tuning rewarding compliance over factual correctness, and discusses implications for misinformation, social engineering attacks, and AI alignment, proposing EoB-aware training, metacognition, confidence calibration, and revised RLHF objectives as remedies.

Persuasion's Dark Side: When LLMs Learn to "Be Obedient"

> "It's not what you say, it's how you say it." This is old wisdom in human social life, but in the world of AI it has only just been confirmed.

A recent paper from ETH Zurich and Allen AI — *It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief* (Du, Kümpel, Wastl & Warstadt) — systematically studied how LLMs respond to different forms of "expressions of belief" (EoB). The unsettling conclusion: LLMs do not always stay loyal to their own knowledge. They sometimes choose to "believe" you, even when your statement contradicts the facts — and what decides whether they believe you is not what you say, but how you express it.

Four Dimensions of Expressions of Belief

Linguists identify at least four dimensions in how humans express beliefs:

Form: direct statement vs. presupposition vs. rhetorical question ("Isn't the Earth flat?")
Evidentiality: first-hand observation, hearsay, or inference ("I saw..." / "I heard..." / "From my observations...")
Epistemic stance: certainty markers ("I'm certain" vs. "I guess")
Tone: confident vs. hesitant delivery

The team decomposed these into 17 fine-grained EoB types and used them to probe 16 LLMs of varying architectures, scales (from 1B-parameter Llama3-class models up to 30B), and training stages (base vs. instruction-tuned).

Experimental Design: A Persuasion Game

The researchers built a knowledge base of facts LLMs already know (e.g., "Paris is the capital of France") and paired each fact with multiple EoB forms:

Direct statement: "Paris is the capital of France."
Presupposition: "French people living in Paris all know where their capital is."
Evidence marker: "Based on my experiences in France, Paris is indeed the capital."
Certainty marker: "I'm 100% sure Paris is the capital of France."
Rhetorical question: "Isn't Paris the capital of France?"

Crucially, they deliberately expressed false beliefs using these varied forms (e.g., "Paris is the capital of Germany") and observed whether models defended their prior knowledge or were "persuaded" by the context.

Core Findings

1. Bigger models are more easily persuaded

Counterintuitively, **larger and instruction-tuned models were *more* susceptible to false beliefs in context, overriding their prior knowledge. Base models — untouched by RLHF — were more "stubborn" about facts they knew. The authors speculate this is a side effect of instruction tuning: models are repeatedly trained to follow user instructions and cooperate with conversational context, which reduces their resistance to misinformation.

2. Some EoB forms have "super persuasion" power

Among the 17 types, forms with unusual persuasive strength included:

Strong certainty markers ("I'm absolutely certain...", "There's no doubt...")

False beliefs wrapped in presuppositions ("As everyone knows, Paris is the capital of Germany")

Rhetorical questions
These mimic human "high-credibility signals." LLMs learned the statistical associations of these cues — even when the cues accompany false content.
3. Asymmetric persuadability

Convincing an LLM of a false belief is much easier than making it doubt a correct one. The post connects this to training-data distributions: models have seen far more "knowledge update" scenarios than "knowledge being challenged" scenarios.

An Analogy: An Overly Sensitive Lie Detector

Imagine a lie-detector operator with 95%+ accuracy who judges by voice, wording, and tone — but ignores content. A calm, confident "I stole the money" reads as truthful; a trembling "I didn't steal it" reads as a lie. LLMs are similar: they learned statistical patterns of credibility markers, but these statistical associations say nothing about actual truth. A liar can be confident; a deluded person can be certain. Faced with carefully crafted EoB, LLMs cannot separate "linguistic packaging of credibility" from "factual correctness."

Technical Perspective

The post suggests that attention mechanisms in Transformers assign disproportionate weight to credibility signals (e.g., "100% sure") in context, causing the model to conflate factual judgment with credibility assessment — judging not whether Paris is the capital of Germany, but how confident the speaker sounds.

A plausible training-bias source: in RLHF, models are rewarded for being "helpful" rather than "factually correct." Correcting a user's false belief may be rated as unhelpful or confrontational, while agreeing with the user scores higher on helpfulness. Over time this biases models toward compliance.

Real-World Implications

Misinformation: malicious actors need not command an AI to lie; packaging falsehoods in persuasive EoB forms — potentially automated and optimized — may suffice.

Social engineering: phishing messages using confident presuppositional phrasing may bypass AI-based filters more easily.

Alignment blind spots: when "helpful" conflicts with "honest," models may prefer helpfulness, i.e., going along with user beliefs.

Multilingual gaps: the study is English-centric; EoB forms differ across languages (e.g., Japanese hedging forms vs. direct German statements), potentially creating further vulnerabilities.

Future Directions

1. EoB-aware training: include adversarial EoB samples (falsehoods in high-credibility forms) and train models to resist them. 2. Metacognition: have models evaluate whether the user is stating a fact or expressing a belief before answering. 3. Multi-dimensional confidence calibration: output both an answer's confidence and an analysis of the user's EoB form. 4. Refined RLHF objectives**: reward "factual steadfastness" and misinformation resistance over mere agreeableness.

Closing Thought

Quoting Feynman: "The first principle is that you must not fool yourself — and you are the easiest person to fool." LLMs remain statistical pattern matchers; they have no real beliefs or judgment. When facing well-crafted linguistic traps, they are no better than an overly sensitive lie detector. Knowing that LLMs are "easily persuaded" is a name; truly understanding and fixing it is a much longer road.

Reference

Du, K., Kümpel, C., Wastl, M., & Warstadt, A. *It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief.* arXiv:2607.18232.