Gricean Retreat: LLMs Know When They're Hallucinating—But Generate Specifics Anyway
Imagine asking a friend about a specific new book. If they don't know it, an honest person says "I don't know it"—but a subtler response retreats to a general one: "I've heard there's a new book getting attention lately." Linguist Paul Grice described this in his 1975 cooperative principle: when uncertain about a specific reference, speakers retreat up the specificity hierarchy—trading informativeness for accuracy. This is called Gricean Retreat.
The question: do large language models do Gricean Retreat?
The Hallucination Dilemma
When asked about entities absent from training data, LLMs classically fabricate—generating plausible but fictional content. The traditional explanation: models don't know what they don't know. But researchers at the University of Colorado Boulder found a surprising answer: the model knows, but it doesn't use that knowledge.
Experimental Design
T-REx Benchmark
The team built a test benchmark on the T-REx knowledge graph, which aligns Wikipedia text with subject-relation-object triples. The key design is synthetic entities: replacing real entities with fictional ones (e.g., swapping "Paris" for "Zarathustra City"), creating entities that could not appear in training data. This lets researchers precisely control which entities fall inside or outside the model's knowledge boundary.
Two Core Questions
For each entity, researchers asked:
1. Do the model's activations encode whether the entity is inside or outside its knowledge boundary? 2. Do the activations predict whether the model is about to generate a specific or generic reference?
Finding 1: Models Do Know Their Knowledge Boundary
Using linear probes across model layers:
- For models above 22B parameters, AUROC exceeds 90% in predicting whether an entity appeared in training data
- Even the smallest models showed (weaker) predictive signal
- Best prediction layers were near the middle of the network
- Stronger prediction under argmax decoding (deterministic decoding forces an earlier "decision")
- Weaker but still significant under multinomial decoding
Interestingly, the "People→Location" relation predicted worst—because its synthetic entities accidentally leaked into the Pile pretraining dataset, so the model "believed" it had seen them.
Finding 2: Models Also Know What They're About to Generate
Model activations reliably predict whether the upcoming reference will be specific or generic:
Finding 3: But Models Don't Retreat
Both signals exist. The critical question: are they linked?
No. In out-of-the-box generation, models overwhelmingly generate specific references regardless of knowledge-boundary status. Even for synthetic entities the model has never seen, it produces specific, plausible-sounding, entirely fabricated references—even when correct generic alternatives are provided. The specificity bias persists across context lengths.
The Essence of the Problem
The paper's deepest insight, in one sentence: "The substrate for Gricean retreat exists, but the strategy for executing it does not."
The model has all the information needed to retreat, but lacks the policy to use it. From a mechanistic interpretability perspective:
1. Knowledge-boundary signals and generation policy are decoupled—two independent internal signals with no connection between them. 2. Hallucination is not a "not knowing" problem, but a "not using what it knows" problem. Models don't need to learn their knowledge boundary—they already know it. They need to learn to retreat when they don't know. 3. Generation policy is trained, not emergent. Models prefer specific references because training (especially RLHF) rewards confident, concrete answers.
Gricean Alignment: A New Direction
The paper proposes Gricean alignment: rather than post-hoc detection and correction of hallucinations, use knowledge-boundary signals during generation to guide the model toward retreating to more generic references. This is not a new model but a new alignment objective—while traditional alignment optimizes for "generating useful content," Gricean alignment targets "retreating to safe phrasing under uncertainty."
This contrasts interestingly with Epanorthosis (LLMs systematically reproducing classical rhetorical self-correction): Epanorthosis is "speak first, revise after" (inference-time intervention); Gricean retreat is "know first, retreat before speaking" (generation-time strategy).
Limitations and Honest Assessment
1. Synthetic entities may not fully represent real scenarios—real unknown entities carry more contextual cues. 2. Only linear probes were tested; nonlinear signals may be stronger. 3. Gricean alignment was not implemented—the paper diagnoses the problem; the framework is conceptual, not a concrete method. 4. Limited model scope—mostly open-source models; closed-source behavior is unknown.
Conceptually, this is another instance of evaluation blind spots: traditional hallucination benchmarks evaluate outputs only, never seeing the gap between internal signals and generation policy. It joins a growing cross-paper consensus—like SOPHIA's residual-stream directions and sycophancy research—that models internally know more than their outputs show; the challenge is activating and using that knowledge.
---
Paper link: https://arxiv.org/abs/2608.13484
HTML version: https://arxiv.org/html/2608.13484v1