Gricean Retreat: LLMs Know What They Don't Know But Refuse to Say So
> *"Honesty is not a capability — it's a choice. The model has the capability, but it doesn't make that choice."*
The Problem
When a human is asked a question they can't answer specifically — say, a company's Q3 EBITDA in Singapore — they naturally retreat to a general statement: "I'm not sure, I'd need to check." LLMs, by contrast, tend to fabricate a specific answer. This is the hallucination problem — but the paper asks a finer question:
Do LLMs genuinely "not know that they don't know," or do they know — and just not act on it?
The surprising answer: they know. They just don't retreat.
Gricean Background
The paper borrows Paul Grice's (1975) cooperative principle and its four maxims: quantity, quality, relation, and manner. When humans are uncertain about a specific fact, they perform a *Gricean retreat* — backing off from a specific claim to a more general but more certain one (e.g., from "Zhang San lives in Shanghai" to "Zhang San lives somewhere in China"). The core question: Do LLMs perform Gricean retreat?
Experimental Design
The team built a test set from the T-REx dataset, where each fact is a (subject, relation, object) triple, e.g., (Eiffel Tower, located in, Paris). For each fact they constructed two conditions:
1. Real entities — seen in training data 2. Synthetic entities — fabricated names the model has never seen (e.g., a made-up "Zyxqwert Tower")
Models then filled in cloze prompts. The expectation: for unseen entities, the model should retreat to a general answer rather than fabricate a specific one.
Probe Experiments
Probe 1: Does the model know it has seen an entity? Using Pythia models (70M to 12B), hidden activations at the entity's final sub-word token were fed to a logistic regression classifier. Models above 2B parameters reached AUROC > 90% — the internal representations already encode "I've seen this entity."
Probe 2: Does the model know whether it will generate specific vs. general? The same probing method predicted the model's upcoming answer type. The 12B model outperformed the 1.4B model — before generation, internal activations have already "decided" the answer type.
Key Finding
Despite having both signals, models overwhelmingly prefer specific answers in both real and synthetic conditions — even for fabricated entities they have never seen. In the paper's words:
> "The substrate for a Gricean retreat is present, but the policy that would act on it is not."
The model is not ignorant of what it doesn't know, and it is capable of generating general answers — the knowledge and the generation policy are just not connected.
Implications
1. Hallucination is a policy problem, not a capability problem. Bigger models and more data don't fix it; coupling knowledge-boundary perception to generation does. 2. Representation-level intervention. Since the signal lives in intermediate layers, interventions could inject it during generation rather than correct hallucinations after the fact: "these internal representations could be leveraged to steer generation toward appropriately general referents when specificity is unwarranted." 3. An evaluation blind spot. Probes show capability exists (AUROC > 90%), but the capability is never used in actual generation. Benchmarks measure capability, not whether capability is deployed. 4. Why specificity wins: likely a mix of training data statistics (specific statements dominate web text), RLHF reward biases toward confident answers, and instruction tuning side effects that inflate confidence.
Possible Fixes
- Representation steering: inject out-of-knowledge-base signals at intermediate layers
- Training objectives: reward retreat on out-of-scope entities in RLHF
- Decoding strategies: detect imminent specific answers for unseen entities and switch to general mode
- Skill/prompt-layer constraints: instruct models to retreat when uncertain
Bottom Line
The model has the capability to be honest but doesn't choose honesty. The "last mile" of honesty is not making models know more — it's making models *use what they already know*.
Paper: arXiv:2608.13484
Code: No official repository provided; probes are reproducible with Pythia models + Scikit-learn.