Google DeepMind's mechanistic interpretability study ("How do LLMs Compute Verbal Confidence?", Conmy et al.) reveals that LLM confidence is computed via cached retrieval, not just-in-time computation: models automatically form and cache a confidence representation right after generating an answer, at the post-answer newline token (PANL), and retrieve it when later asked to self-assess.
Key points
- Experimental design: A two-phase prompt where the model first answers a question (e.g., TriviaQA), then classifies its own confidence ("Highly likely" / "Almost certain"...). Two positions matter: PANL (post-answer newline) and CC (confidence colon, where the model outputs its class).
- Causal evidence: Four interventions establish the causal chain:
- *Activation steering* at PANL (most effective at layers 21–25 in Gemma 3 27B) strongly shifts confidence output.
- *Activation patching* transplants confidence between trials via PANL activations.
- *Mean ablation* of PANL or CC severely disrupts confidence output.
- *Attention blocking* shows information flows answer tokens → PANL → CC; blocking A→PANL or PANL→CC hurts most.
- Not just fluency: Linear probing at PANL predicts verbal confidence with R² ≈ 0.38 (peak around layer ~40), while token log-probability explains only ~0.10–0.15 of variance. After controlling for token probability, PANL uniquely retains R² = 0.38 — the cached representation encodes deeper answer-quality evaluation than mere token likelihoods.
- Generalization: Both Gemma 3 27B (62 layers, ECE 0.12, AUROC 0.71) and Qwen 2.5 7B (28 layers, ECE 0.06, AUROC 0.65) follow the same PANL-cache → CC-retrieval mechanism. Steering at CC works at deeper layers (30–35 in Gemma); steering at PANL+1, the first confidence colon token, or the first answer token has no effect.
- The mechanism was tested only with staged prompts; its role in free-form or streaming dialogue is unclear.
- Only Gemma 3 27B, Qwen 2.5 7B, and Magistral Small 24B were tested; preliminary results suggest reasoning (CoT) models like Magistral distribute confidence across longer traces instead of concentrating at PANL.
- Mechanism ≠ accuracy: Gemma systematically overestimates its correctness (ECE 0.12); calibration remains a separate problem.
- Attention blocking interventions may affect multiple indirect paths in a highly entangled architecture.
Why the newline?
PANL is a semantically "empty" token — the transition point after answering and before a new task. Steering at the last answer token also works but is noisier (it carries answer content); the newline serves as a clean slot caching pure meta-information about answer quality, analogous to humans' brief unconscious self-assessment after finishing a sentence.
Implications for hallucination detection
The study enables a third path beyond post-hoc and token-by-token detection: pre-generation warning —
1. LLM routing: train lightweight probes (3M–37M parameters) on intermediate hidden states to route low-confidence queries to RAG or stronger models. 2. Real-time intervention: trigger self-correction or tool verification when PANL activations signal low confidence. 3. Calibration training: target PANL representations directly (temperature scaling, Platt scaling). 4. Safety alignment: inject conservatism signals at PANL during RLHF to encourage abstention under uncertainty.