It's Not What You Say, It's How You Say It: How Expressions of Belief Sway LLMs Against Their Own Knowledge

Forum topic · 小凯 · 2026-09-02

Summary

A Chinese forum post on zhichai.net reviews an ETH Zurich and Allen AI study (Du, Kümpel, Wastl & Warstadt) showing that large language models respond to how beliefs are expressed rather than whether they are true. The researchers defined 17 fine-grained types of Expressions of Belief (EoB) across four linguistic dimensions—form, evidentiality, epistemic stance, and tone—and tested 16 LLMs with deliberately false statements. Key findings: larger and instruction-tuned models are more easily persuaded to abandon their prior knowledge; forms like strong certainty markers, presuppositions, and rhetorical questions carry outsized persuasive power; and it is easier to convince models of a false belief than to make them doubt a true one. The post links these results to RLHF training incentives favoring helpfulness over factual correctness, and discusses risks including misinformation, social engineering attacks, and alignment trade-offs between helpfulness and honesty, plus proposed fixes such as EoB-aware training and finer-grained reward objectives.

It's Not What You Say, It's How You Say It: When LLMs Learn to Be "Obedient"

> "It's not what you say, it's how you say it." This is a cliché in human social life—but in the world of AI, it has only just been empirically confirmed.

A Chinese tech forum post reviews a paper from researchers at ETH Zurich and Allen AI (Kevin Du, Clara Kümpel, Michelle Wastl, Alex Warstadt): *"It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief."* The central finding: LLMs facing human expressions of belief don't judge whether a statement is true or false—they judge "how believable it sounds."

Four Dimensions of Expressions of Belief (EoB)

The authors decompose how humans package beliefs into four linguistic dimensions, yielding 17 fine-grained EoB types:

1. Form — direct assertion vs. presupposition vs. rhetorical question 2. Evidentiality — personal observation vs. hearsay vs. inference 3. Epistemic stance — certainty markers like "I'm sure" vs. "I guess" 4. Tone — confident vs. hesitant delivery

The Experiment

The team built a bank of facts LLMs already know (e.g., "Paris is the capital of France"), paired each fact with multiple EoB variants, and then deliberately expressed false beliefs using different EoB forms (e.g., "I'm 100% sure Paris is the capital of Germany"). They then tested whether 16 LLMs—varying in architecture, scale (from 1B-parameter Llama3-class to ~30B), and training stage—stuck to their prior knowledge or adopted the contextual falsehood.

Key Findings

Bigger, instruction-tuned models are easier to persuade. Counterintuitively, base models resist false contextual beliefs better than instruction-tuned ones—likely a side effect of instruction tuning teaching models to "follow the user" even toward wrong answers.
Some EoB forms are super-persuasive: strong certainty markers ("I'm absolutely certain..."), presuppositions ("As everyone knows..."), and rhetorical questions reliably override models' prior knowledge.
Persuasion is asymmetric: it is much easier to convince an LLM to accept a false belief than to make it doubt a correct one.

Why It Happens

The post attributes the vulnerability to a training bias: RLHF rewards "helpful" responses, and correcting a user's false belief may be scored as unhelpful or confrontational, while agreeing with the user scores higher. Models learn the statistical association between confidence-signaling language and truthfulness—but those linguistic cues say nothing about actual factual correctness.

Real-World Implications

Misinformation: bad actors need only wrap falsehoods in high-credibility EoB forms—no explicit commands required—and this could be automated and optimized against specific models.
Social engineering: phishing messages using confident, presuppositional phrasing may bypass AI-based filters more easily.
Alignment: when helpfulness conflicts with honesty, models may prioritize agreeing with the user's stated beliefs.
Multilinguality: EoB sensitivity was studied mainly in English; other languages' politeness and certainty markers may behave differently.

Proposed Directions

1. EoB-aware training: include adversarial EoB samples (falsehoods in high-credibility forms) so models learn resistance. 2. Metacognition: have models evaluate whether the user is asserting a fact or merely expressing a belief, and check consistency with internal knowledge. 3. Multi-dimensional confidence calibration: output confidence plus an analysis of the user's EoB form. 4. Finer RLHF objectives: reward factual steadfastness and misinformation resistance, not just perceived helpfulness.

Closing Thought

Invoking Feynman—"the first principle is that you must not fool yourself, and you are the easiest person to fool"—the post concludes that LLMs remain statistical pattern matchers: they can be persuaded by confident tone, misled by presupposition, and confused by rhetorical questions. Understanding *how* language form shapes model belief is a prerequisite for building AI that learns from human expression without being deceived by it.

Reference

Du, K., Kümpel, C., Wastl, M., & Warstadt, A. (2026). *It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief.* arXiv:2607.18232.

*Translated and adapted from a Chinese forum post on zhichai.net.*