Hallucinations Are Not Errors—They Are Confident Errors
> arXiv 2026 | Google Research + Tel Aviv University > > Gal Yona's team: if a model stays silent whenever uncertain, every error you eliminate costs you half a correct answer. This is not a technical flaw—it is mathematics.
A Question: Would You Trust a Doctor Who Never Says "I Don't Know"?
Ask a model: "How many photographs are in Isa Genzken's Der Spiegel?"
It answers: "121."
You believe it, check, and find it's wrong. You call it a hallucination.
But if it said: "I'm not entirely sure, but my best guess is 121"—would you still blame it?
Yona's team argues: no. In the second case, the model didn't lie. It laid its assumption on the table. The error remains, but its nature changes—from "deception" to "uncertain speculation."
This is their redefinition of hallucination:
> Hallucination ≠ error. Hallucination = a confident error.
An error with appropriate hedging ("I'm not sure," "this may be," "to my knowledge") is not a hallucination—it is a hypothesis for your consideration.
The Binary Dilemma: Shut Up or Make Things Up
Existing hallucination-mitigation methods boil down to one move: abstain when uncertain. But this has a hidden cost.
The Utility Tax
Using AUROC, Yona's team ran the numbers (baseline error 25% → target 5%):
| Model's ability to distinguish right from wrong | Cost | |:---|:---| | AUROC = 0.71 (literature average) | Discard 52% of correct answers | | AUROC = 0.85 (literature ceiling) | Discard ~28% of correct answers | | AUROC ≥ 0.95 | Utility tax < 5%, negligible |
The last row is an ideal that no method currently achieves.
If you insist on "zero hallucination," you must accept a model that stays silent on more than half the questions it could answer correctly. To eliminate errors, you must also eliminate correct answers.
A Third Path: Faithful Uncertainty
> Faithful Uncertainty: align the model's verbalized uncertainty (what it says) with its internal uncertainty (what it actually believes).
Calibration vs. Faithfulness
| | Calibration | Faithful Uncertainty | |:---|:---|:---| | Asks | "On average, does my confidence match accuracy?" | "Does this specific answer's hedge reflect my true current state?" | | Granularity | Aggregate, distribution-level | Instance-level guarantee | | Counterexample | Giving every answer 0.6 confidence when 60% are correct = perfectly calibrated, but totally uninformative | Even with poor aggregate calibration, each hedge faithfully reflects internal state |
Calibration is statistics. Faithful uncertainty is honesty.
Metacognition: Self-Knowledge Is Feasible
Metacognition = awareness of one's own uncertainty + the ability to act on it.
A counterintuitive claim: a model cannot know external truth (an unbounded problem), but it can know its internal confidence (a fully observable, closed-loop problem)—like someone who doesn't know if they're right but can feel whether they're guessing.
Behavioral Semantics
- Model says "I'm confident" → repeat the question, it likely gives the same answer
- Model says "I'm not sure" → repeat the question, it may give a different answer
- Efficiency: retrieve only when confidence is low, avoiding systematic tool overuse
- Reliability: express skepticism when retrieved evidence conflicts with internal priors
- Gemini 2.5 Pro, GPT-5, o3, DeepSeek R1, Claude Opus 4… all fall along the diagonal
- High factuality = high abstention = moving left (paying the utility tax)
- The upper-right corner (high factuality + high utility) is completely empty
- Extended reasoning increases hallucinations (optimizing utility over abstention)
- Extended reasoning decreases abstention ability (incentivized to complete reasoning chains)
- The performance gap between answerable and unanswerable questions widens (the model learns to rationalize wrong answers)
- Yona G, Geva M, Matias Y. (2026). *Hallucinations Undermine Trust; Metacognition is a Way Forward*. arXiv:2605.01428v1 [cs.CL]. https://arxiv.org/abs/2605.01428
This internal-state-to-language mapping is the operational definition of faithful uncertainty.
Agents: Metacognition as a Control Layer
An agent without metacognition is like a blindly flying plane—the harness makes all decisions via heuristics. With metacognition, agents gain a dynamic adjustment layer:
Tools don't remove the need for metacognition—they amplify it. After ten search results, you need to know which ones to trust.
Empirics: All Frontier Models Sit on the Diagonal
Using SimpleQA Verified (Figure 3): X-axis = factuality, Y-axis = utility, color = abstention rate.
This illustrates the Discrimination Gap: a fundamental limitation of the current paradigm, not of any single model.
Worse, reasoning models:
Methods: How to Train Metacognition?
| Path | Technique | Challenge | |:---|:---|:---| | Metacognitive prompting | Teach the model to self-assess before generating | Prompt engineering ceiling | | Supervised fine-tuning | Teach hedging expression | Bootstrapping paradox: static labels vs. dynamic knowledge boundaries | | Internal representation steering | Guide via internal representations | Unstable representation-behavior mapping | | RL + intrinsic signals | Use internal uncertainty as reward | Signal erased during alignment |
The Bootstrapping Paradox
Training "I don't know" requires labeled data—but uncertainty labels are relative to the model's current knowledge, not objective. "X is outside model A's training set" may be something larger model B already knows; B saying "I don't know" about X would itself be uncertainty hallucination. This demands dynamic dataset infrastructure or new bootstrapping approaches.
Signal Preservation
Pretrained models have decent internal uncertainty representations, but alignment techniques like RLHF systematically destroy them—aligned models are significantly more overconfident than base models. "Uncertainty-preserving" alignment algorithms are needed.
Limitations
| Issue | Note | |:---|:---| | Mostly theoretical | Conceptual framework, not extensive new experiments | | Implementation gap | "Faithful uncertainty remains an unfilled gap" | | Causal evaluation is hard | Model may learn "always hedge on rare entities" rather than genuinely sensing internal state | | User acceptance | Some settings genuinely favor confident answers (emergency medicine, creative writing) |
Core Conclusions
1. Hallucination = confident error. An appropriately hedged error is not a hallucination—it's a hypothesis. 2. Calibration ≠ discrimination. Average confidence matching accuracy is useless; instance-level faithfulness is what matters. 3. The utility tax is mathematics. At AUROC 0.71, eliminating errors costs 52% of correct answers. 4. Metacognition is feasible. Models can't know external truth, but can know their own confidence—a closed-loop problem. 5. Agents need a metacognitive control layer. Blind flight is unsustainable. 6. All current frontier models sit on the diagonal. The upper-right corner is empty—the field's open challenge.