> arXiv 2026 | Google Research + Tel Aviv University > > Gal Yona's team: if a model stays silent whenever it is uncertain, every eliminated error costs you half a correct answer. This is not a technical flaw—it is math.
A Question: Would You Trust a Doctor Who Never Says "I Don't Know"?
Ask a model: "How many photos are in Isa Genzken's *Der Spiegel*?"
It answers: "121."
You trust it, check, find it's wrong, and call it a hallucination.
But if it had said: "I'm not very sure, but my best guess is 121"—would you still blame it?
Yona's team argues you would not. In the latter case the model isn't lying; it lays its assumption out for you to see. The error remains, but its nature changes—from deception to uncertain conjecture.
This is their redefinition:
> Hallucination ≠ error. Hallucination = confident error.
An error with appropriate hedging ("I'm not sure," "this might be," "as far as I know") is not a hallucination—it's an assumption offered for reference.
The Binary Dilemma: Abstain or Confabulate
Existing anti-hallucination methods mostly rely on one trick: abstain when uncertain. Saying "I don't know" beats confabulation—but there's a hidden cost.
The Utility Tax
Measured via AUROC:
| Discrimination ability | Baseline error 25% → target 5% | Cost | |:---|:---|:---| | AUROC = 0.71 (literature average) | compress errors to 5% | 52% of correct answers discarded | | AUROC = 0.85 (literature ceiling) | same | ~28% discarded | | AUROC ≥ 0.95 | same | utility tax < 5%, negligible |
The last row is an ideal no current method achieves. Insisting on "zero hallucination" means accepting silence on more than half of the questions the model *could* answer. To eliminate errors, you must also eliminate correctness.
The Third Path: Faithful Uncertainty
> Faithful Uncertainty: align the model's *verbalized* uncertainty (what it says) with its *internal* uncertainty (what it actually believes).
Calibration vs. Faithfulness: A Key Distinction
| | Calibration | Faithful Uncertainty | |:---|:---|:---| | Question asked | "Does my confidence match accuracy on average?" | "Does the hedging on this specific answer reflect my true state right now?" | | Granularity | Aggregate property over the distribution | Instance-level guarantee, per answer | | Counterexample | All answers at 0.6 confidence with 60% accuracy = perfectly calibrated, but zero discrimination | Even with poor aggregate calibration, each hedge truthfully reflects internal state |
Calibration is statistics. Faithful uncertainty is *honesty*.
Metacognition: Self-Knowledge Is Feasible
Metacognition = awareness of one's own uncertainty + the ability to act on it.
The counterintuitive claim:
> A model cannot know external ground truth (an unbounded problem), but it can know its internal confidence (a fully observable, closed-loop problem).
Behavioral Semantics
- Model says "I'm confident" → asked again, it likely gives the same answer
- Model says "I'm unsure" → asked again, it may give a different answer
- Efficiency: retrieve only when confidence is low, avoiding systematic tool over-use
- Reliability: express skepticism when retrieved evidence conflicts with internal priors, rather than blindly obeying
- Gemini 2.5 Pro, GPT-5, o3, DeepSeek R1, Claude Opus 4… all frontier models fall along a diagonal
- High factuality = high abstention = moving left (paying the utility tax)
- The upper-right corner (high factuality + high utility) is completely empty
- Extended reasoning increases hallucinations (optimizing utility over abstention)
- Extended reasoning decreases abstention ability (incentivized to complete reasoning paths rather than stop)
- The performance gap between answerable and unanswerable questions widens (the model learns to rationalize wrong answers)
- Yona G, Geva M, Matias Y. (2026). *Hallucinations Undermine Trust; Metacognition is a Way Forward*. arXiv:2605.01428v1 [cs.CL]. https://arxiv.org/abs/2605.01428
This mapping from internal state to external language is the operational definition of faithful uncertainty.
Agents: Metacognition as a Control Layer
An agent without metacognition is like an aircraft flying blind—the harness makes all decisions via heuristics. With metacognition, the agent gains a dynamic regulation layer:
Tools don't remove the need for metacognition—they amplify it. After ten searches, you need to know which of the ten results to trust.
Empirics: All Frontier Models Lie on the Diagonal
Using SimpleQA Verified (Figure 3): X-axis = factuality (attempted accuracy), Y-axis = utility (accuracy), color = abstention rate.
This visualizes the Discrimination Gap: a fundamental limitation of the current paradigm, not of any single model.
Worse, for reasoning models:
Methods: How to Train Metacognition?
| Path | Technique | Challenge | |:---|:---|:---| | Metacognitive prompting | Teach the model to self-assess before generating | Prompt-engineering ceiling | | Supervised fine-tuning | Teach the model to express uncertainty | Bootstrapping paradox: static labels vs. dynamic knowledge boundaries | | Internal representation steering | Guide behavior from internal representations | Unstable representation–behavior mapping | | RL + intrinsic signals | Use internal uncertainty as reward | Signal erased during alignment |
The Bootstrapping Paradox
Training "I don't know" requires labeled data, but uncertainty labels are relative to a model's current knowledge, not objective. Teach model A "I don't know X" → A learns "X isn't in my training set"—but a larger model B may already know X, making B's "I don't know" a *hallucinated uncertainty*. This requires dynamic dataset infrastructure or entirely new bootstrapping methods.
Signal Preservation
Pretrained models have decent internal uncertainty representations, but alignment techniques like RLHF systematically destroy these signals—aligned models are significantly more overconfident than base models. "Uncertainty-preserving" alignment algorithms are needed.
Limitations
| Issue | Note | |:---|:---| | Mostly theoretical | Conceptual framework and analysis, not extensive new experiments | | Implementation gap | "Faithful uncertainty remains an unbridged gap" | | Causal evaluation is hard | Models may learn to "always hedge on rare entities" rather than truly sensing internal state | | User acceptance | Some users genuinely prefer confident answers (medical emergencies, creative writing) |
Core Conclusions
1. Hallucination = confident error. A hedged error is not a hallucination but an assumption. 2. Calibration ≠ discrimination. Aggregate confidence matching accuracy is useless; per-instance faithfulness is what matters. 3. The utility tax is math. At AUROC 0.71, eliminating errors costs 52% of correct answers. 4. Metacognition is feasible. Models can't know external truth, but can know their own confidence—a closed-loop problem. 5. Agents need a metacognitive control layer. Blind flight is unsustainable. 6. All frontier models sit on the diagonal. The upper-right corner is empty—the open problem of the field.