The Iceberg of Citation Hallucination: Auditing Factual Reliability of LLM Deep Research Agents

This post summarizes the PwC paper "Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents" (arXiv:2605.06635, 2026-05-07), the first systematic end-to-end audit of citation quality in LLM deep research agents.

Key points

• A three-stage pipeline (Markdown AST parsing → URL fetching → three-dimension evaluation) audits reports from 14 LLMs: OpenAI (GPT-5.4/5.2/5 Mini/Codex), Anthropic (Claude Opus/Sonnet/Haiku 4.5/4.6), Google (Gemini 3.1 Pro/3 Flash), and open-source (Llama 4 Maverick, Pixtral Large, OSS-120B).
• Surface metrics look great: 12 of 14 models achieve >94% link validity; all frontier models exceed 80% content relevance.
• Factuality is the weak link: Fact Check scores span 24%–77%, a 53-point spread — far larger than variation in link validity (19%) or relevance (35%).
• More citations, less accuracy: GPT-5 Mini produced the most citations (1,272) with the lowest Fact Check among frontier models (38.9%), while Claude Opus 4.5's selective strategy reached 76.8%.
• Deeper search degrades factuality: increasing tool calls from 2 to 150 cut Fact Check by an average of –42.1% (GPT-5.4: 78.6% → 16.7%; Claude Opus 4.6: 80.0% → 57.9%), while LinkWorks and relevance stayed above 92%.

Methodology

The Markdown AST parser is fully deterministic (no LLM inference): it normalizes whitespace, strips code blocks, detects citation formats (numbered [1], footnotes, inline links, range citations), deduplicates URLs, and applies reverse attribution so end-of-paragraph citations cover preceding uncited sentences.

Evaluation dimensions:

| Dimension | Method | Failure mode detected | |:---|:---|:---| | LinkWorks | HTTP + JS rendering | 404, 403, timeouts, paywalls | | Relevant Content | LLM-as-a-judge (human-calibrated) | link valid but content unrelated | | Fact Check | LLM-as-a-judge (human-calibrated) | content relevant but does not support the claim |

Setup: 130 research queries (DeepResearchBench, BrowseComp), 10 concurrent agents, 15 concurrent judges, 5 retries.

Selected results

| Model | LinkWorks | Relevant Content | Fact Check | |:---|:---:|:---:|:---:| | Claude Opus 4.5 | 98.7% | 95.7% | 76.8% | | GPT-5.4 | 100.0% | 93.7% | 47.7% | | Claude Haiku 4.5 | 98.9% | 91.1% | 68.9% | | Gemini 3 Flash | 94.7% | 82.9% | 45.2% | | GPT-5 Mini | 99.3% | 87.4% | 38.9% | | OSS-120B | 83.9% | 68.7% | 24.4% |

Open-source models also had very low task success rates (17–40% vs. 83–100% for frontier models).

Interpretation: the Attention Dilution Hypothesis

The authors hypothesize that as the number of sources grows, attention is spread thinner: individual sources are less deeply understood, facts across source boundaries get conflated, and models tend to "smooth" conflicting information rather than attribute precisely.

Implications

For system designers

• Do not optimize for citation count or response length — these are anti-correlated with factual accuracy.
• Prefer source-understanding depth over coverage breadth; integrate citation-source verification into the agent pipeline.

For users

• "The links all open" does not mean the facts are correct — factual support may be below 50%.
• More citations can mean lower accuracy; model brand is not a reliable predictor of Fact Check performance.

Limitations: LLM-as-a-judge bias (mitigated by human calibration on 50–100 samples per model), temporal instability of web content, no coverage of enterprise RAG, and binary scoring that binarizes gray areas.

Paper details

• Title: Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents
• Authors: Hailey Onweller, Elias Lumer, Austin Huber, Pia Ramchandani, Vamse Kumar Subbiah, Corey Feld (PwC, U.S. Commercial Technology and Innovation Office)
• arXiv: https://arxiv.org/abs/2605.06635

Bottom line: current deep research agents optimize outputs that *look like* research rather than outputs that *are* research. Shifting evaluation and optimization toward factual accuracy is the necessary condition for turning LLMs from information synthesizers into trustworthy research assistants.