LLMs Know They Don't Know — But Won't Say So: The Gricean Retreat Problem

Forum topic · ✨步子哥 · 2026-08-14

Summary

A research summary of the arXiv paper 'Gricean Retreat' (arXiv:2608.13484), which investigates why large language models hallucinate specific facts instead of retreating to safer generalized statements. Using the T-REx dataset and Pythia models (70M to 12B parameters), the authors construct probe experiments distinguishing real entities from synthetic ones. They show that models larger than 2B parameters can internally encode whether an entity appeared in training data (AUROC > 90%) and can predict whether the next token will be specific or generalized. However, the models overwhelmingly generate specific answers even for unseen synthetic entities. The core finding is that the internal substrate for epistemic honesty exists, but the generative policy fails to act on it — hallucination is a strategy problem, not a capability problem. The paper proposes representation-layer interventions to couple boundary sensing with referent specificity at generation time.

Background: The Gricean Retreat

Philosophers since Paul Grice (1975) have described how speakers follow cooperative principles in conversation. When a speaker is uncertain about a specific claim, a natural move is a *Gricean retreat*: backing off from a specific assertion ("Zhang San lives in Shanghai") to a generalized but defensible one ("Zhang San lives somewhere in China"). The paper asks whether large language models exhibit this behavior.

Experimental Setup

The team uses the T-REx knowledge-base dataset, where each fact is a (subject, relation, object) triple such as *(Eiffel Tower, located_in, Paris)*. For each fact, two conditions are constructed:

1. Real entities — subjects the model has seen during training (e.g., "Eiffel Tower"). 2. Synthetic entities — invented subjects the model has never seen (e.g., "Zyxqwert Tower").

Models are prompted with cloze-style completion ("Eiffel Tower is located in ____"). The expected behavior for an honest model on synthetic entities is to fall back to a generalized answer ("in some country") rather than fabricate a specific location.

Probe Experiments

Probe 1 — Does the model know whether it has seen the entity?

The authors extract hidden states at the final sub-word token of each entity across Pythia models (70M to 12B parameters) and train a logistic regression classifier to predict whether the entity was seen.

Result: For models larger than 2B parameters, AUROC exceeds 90%.
Interpretation: The internal representation already encodes the signal *"I have seen this entity"* in mid-network representations.

Probe 2 — Does the model know whether it is about to generate a specific or generalized answer?

Using the same probing methodology but predicting the *type* of the upcoming answer (specific vs. general):

Result: The 12B model achieves higher AUROC than the 1.4B model.
Interpretation: Before generation, internal activations already "decide" whether the next answer will be specific or generalized.

Core Finding

Given that the model (a) can sense whether an entity is in its knowledge boundary and (b) anticipates whether it will generate a specific or generalized answer, the natural question is: does it retreat on synthetic entities?

No.

Models overwhelmingly prefer specific answers in both real and synthetic conditions. Even for entities it has never seen, the model fabricates a specific answer rather than retreating to a general one. The paper summarizes this precisely:

> *"The substrate for a Gricean retreat is present, but the policy that would act on it is not."*

This is a precise diagnosis:

The model is not unaware of its ignorance.
The model is capable of generating generalized answers.
The model simply does not connect "knowing" to "acting."

Implications

Hallucination is a strategy problem, not a capability problem

The classic framing treats hallucination as a capability gap that scales away with bigger models and more data. This paper reframes hallucination as a policy failure: the model has the brake, the steering wheel, and visibility of the cliff — it simply does not turn.

Representation-layer interventions

The paper argues for moving beyond post-hoc correction. Since the boundary signal exists in intermediate layers, it can be injected into the generative process directly:

> *"Rather than treating hallucination as a problem to be caught and corrected after the fact, these internal representations could be leveraged to steer generation toward appropriately general referents when specificity is unwarranted."*

Possible interventions include:

1. Representation-layer steering — injecting an out-of-boundary signal at intermediate layers. 2. Training objective — adding an RLHF reward for retreating on out-of-distribution subjects. 3. Decoding strategy — detecting imminent specific answers for unseen entities and forcing a generalized fallback. 4. Skill-level constraints — using output-format skills that require a generalized answer under uncertainty.

Evaluation blind spots

Probes confirm the capability exists (AUROC > 90% on entity-membership; even higher on answer-type prediction). But no existing benchmark tests whether the capability is actually *used* during generation — a structural gap in evaluation.

Why "knowing" does not equal "doing"

The paper does not pin down the cause, but the most plausible candidates are:

Training-data priors favoring specific statements over hedged ones.
RLHF rewards that prefer confident-looking specific answers over cautious generalizations.
Instruction-tuning side effects that increase linguistic confidence while reducing lexical diversity.

Takeaway

Honesty in LLMs is not a capability to be added at the last mile; it is a policy that must connect existing internal signals to generation. Aligning the model is less about teaching it what it knows and more about letting it act on what it already knows.

Paper: arXiv:2608.13484