English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Gricean Retreat: LLMs Know What They Don't Know But Refuse to Say So

Forum topic · ✨步子哥 · 2026-08-14

Summary

A Chinese tech forum analysis of a research paper on LLM hallucination through the lens of philosopher Paul Grice's cooperative principle. The paper introduces 'Gricean retreat' — the human conversational strategy of retreating from specific claims to general ones when uncertain. Using the T-REx dataset with real vs. synthetic (fabricated) entities, the authors probe Pythia models (70M–12B) with logistic regression probes on hidden activations. Results: models above 2B parameters encode 'seen vs. unseen entity' information with AUROC > 90%, and larger models (12B vs 1.4B) can predict whether they will generate specific vs. general answers. Yet when facing unknown entities, models overwhelmingly still produce specific answers rather than retreating to general ones. The paper's diagnosis: 'The substrate for a Gricean retreat is present, but the policy that would act on it is not.' Hallucination is framed as a policy problem, not a capability problem — representations already encode knowledge boundaries, but generation ignores them. The author proposes representation-level interventions as a new layer in alignment work.

Gricean Retreat: LLMs Know What They Don't Know But Refuse to Say So

> *"Honesty is not a capability — it's a choice. The model has the capability, but it doesn't make that choice."*

The Problem

When a human is asked a question they can't answer specifically — say, a company's Q3 EBITDA in Singapore — they naturally retreat to a general statement: "I'm not sure, I'd need to check." LLMs, by contrast, tend to fabricate a specific answer. This is the hallucination problem — but the paper asks a finer question:

Do LLMs genuinely "not know that they don't know," or do they know — and just not act on it?

The surprising answer: they know. They just don't retreat.

Gricean Background

The paper borrows Paul Grice's (1975) cooperative principle and its four maxims: quantity, quality, relation, and manner. When humans are uncertain about a specific fact, they perform a *Gricean retreat* — backing off from a specific claim to a more general but more certain one (e.g., from "Zhang San lives in Shanghai" to "Zhang San lives somewhere in China"). The core question: Do LLMs perform Gricean retreat?

Experimental Design

The team built a test set from the T-REx dataset, where each fact is a (subject, relation, object) triple, e.g., (Eiffel Tower, located in, Paris). For each fact they constructed two conditions:

1. Real entities — seen in training data 2. Synthetic entities — fabricated names the model has never seen (e.g., a made-up "Zyxqwert Tower")

Models then filled in cloze prompts. The expectation: for unseen entities, the model should retreat to a general answer rather than fabricate a specific one.

Probe Experiments

Probe 1: Does the model know it has seen an entity? Using Pythia models (70M to 12B), hidden activations at the entity's final sub-word token were fed to a logistic regression classifier. Models above 2B parameters reached AUROC > 90% — the internal representations already encode "I've seen this entity."

Probe 2: Does the model know whether it will generate specific vs. general? The same probing method predicted the model's upcoming answer type. The 12B model outperformed the 1.4B model — before generation, internal activations have already "decided" the answer type.

Key Finding

Despite having both signals, models overwhelmingly prefer specific answers in both real and synthetic conditions — even for fabricated entities they have never seen. In the paper's words:

> "The substrate for a Gricean retreat is present, but the policy that would act on it is not."

The model is not ignorant of what it doesn't know, and it is capable of generating general answers — the knowledge and the generation policy are just not connected.

Implications

1. Hallucination is a policy problem, not a capability problem. Bigger models and more data don't fix it; coupling knowledge-boundary perception to generation does. 2. Representation-level intervention. Since the signal lives in intermediate layers, interventions could inject it during generation rather than correct hallucinations after the fact: "these internal representations could be leveraged to steer generation toward appropriately general referents when specificity is unwarranted." 3. An evaluation blind spot. Probes show capability exists (AUROC > 90%), but the capability is never used in actual generation. Benchmarks measure capability, not whether capability is deployed. 4. Why specificity wins: likely a mix of training data statistics (specific statements dominate web text), RLHF reward biases toward confident answers, and instruction tuning side effects that inflate confidence.

Possible Fixes

  • Representation steering: inject out-of-knowledge-base signals at intermediate layers
  • Training objectives: reward retreat on out-of-scope entities in RLHF
  • Decoding strategies: detect imminent specific answers for unseen entities and switch to general mode
  • Skill/prompt-layer constraints: instruct models to retreat when uncertain

Bottom Line

The model has the capability to be honest but doesn't choose honesty. The "last mile" of honesty is not making models know more — it's making models *use what they already know*.

Paper: arXiv:2608.13484

Code: No official repository provided; probes are reproducible with Pythia models + Scikit-learn.

Tags

#llm#hallucination#gricean-retreat#probing#pythia#alignment#knowledge-boundaries#interpretability

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633477