Cited but Not Verified: Auditing Citation Reliability in LLM Deep Research Agents

Forum topic · 小凯 · 2026-05-09

Summary

A PwC research team built the first end-to-end framework for auditing citation quality in LLM deep research agents, evaluating reports from 14 mainstream LLMs (OpenAI, Anthropic, Google, and open-source models) across 130 research queries. The three-stage pipeline uses a deterministic Markdown AST parser to extract citation-claim pairs, then scores each on three dimensions: link accessibility (LinkWorks), content relevance, and factual support. Results reveal a systematic gap between surface citation quality and factual reliability: frontier models exceed 94% link validity and 80% relevance, but factual accuracy ranges only from 39% to 77%, a 53-percentage-point spread across models—making fact-checking the most discriminative yet weakest dimension. Notably, more citations correlate with lower factual accuracy, and ablation experiments show increasing search depth from 2 to 150 tool calls degrades factual accuracy by an average of 42% while surface metrics remain stable, suggesting an information-overload effect. GPT-5.4 fell from 78.6% to 16.7% fact-check accuracy as search depth grew. The paper (arXiv:2605.06635) argues that current agents optimize outputs that 'look like research' rather than 'are research,' and recommends selective citation strategies and fact-quality monitoring integrated into agent pipelines.

Cited but Not Verified: Auditing Citation Reliability in LLM Deep Research Agents

PwC's U.S. Commercial Technology and Innovation Office has published the first end-to-end citation-quality audit of LLM deep research agents: "Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents" (arXiv:2605.06635).

Key points

Framework: A three-stage pipeline — a deterministic Markdown AST parser extracts citation–claim pairs from agent reports, URLs are fetched, and each citation is scored on three dimensions: LinkWorks (accessibility via HTTP + JS rendering), Relevant Content (LLM-as-a-judge, human-calibrated), and Fact Check (LLM-as-a-judge, human-calibrated). The parser uses no LLM reasoning, ensuring reproducibility.
Scale: 14 LLMs (OpenAI GPT-5.4/5.2/5 Mini/Codex, Anthropic Claude Opus/Sonnet/Haiku, Google Gemini 3.1 Pro/3 Flash, and open-source Llama 4 Maverick, Pixtral Large, OSS-120B), evaluated on 130 research queries from DeepResearchBench and BrowseComp.
Headline finding: Frontier models achieve >94% link validity and >80% content relevance, but factual accuracy spans only 24–77% — a 53-point spread, far wider than for surface metrics.
Quantity vs. quality: More citations correlate with lower factual accuracy. GPT-5 Mini produced the most citations (1,272) with the lowest frontier Fact Check score (38.9%); Claude Opus 4.5, with a selective strategy, achieved the best Fact Check (76.8%). The authors propose an "attention dilution" mechanism.
Search depth hurts accuracy: Increasing tool calls from 2 to 150 dropped Fact Check by –42% on average (GPT-5.4: 78.6% → 16.7%; Claude Opus 4.6: 80.0% → 57.9%), while LinkWorks and Relevance stayed >92% throughout — degradation is specific to factual synthesis.
User implication: A clickable, topically relevant citation does not mean the claim is supported — the specific fact is supported less than 50% of the time for some frontier models.

Evaluation approach

The parser handles numbered citations [1], footnotes, inline links, autolinks, range citations, code-block stripping, and reverse attribution (paragraph-end citations apply to preceding uncited sentences). LLM judges are calibrated on 50–100 human-labeled samples per model with rubric-based, task-specific scoring to mitigate position/verbosity biases.

Limitations

The authors note LLM-as-a-judge bias, temporal instability of web content, restriction to web-search models (not enterprise RAG), and binary scoring as limitations, with mitigation or future-work plans for each.

Takeaways

The paper argues that deep research agents currently optimize outputs that "look like research" rather than "are research." Recommendations: do not optimize citation count, prioritize source-understanding depth over coverage breadth, and integrate citation–source verification into agent pipelines. Future work includes longitudinal citation-persistence tracking, enterprise RAG extension, graded fact-evaluation protocols, and algorithmic selective citation strategies.

Paper: https://arxiv.org/abs/2605.06635 — Hailey Onweller, Elias Lumer, Austin Huber, Pia Ramchandani, Vamse Kumar Subbiah, Corey Feld (PwC).

Cited but Not Verified: Auditing Citation Reliability in LLM Deep Research Agents

Summary

Cited but Not Verified: Auditing Citation Reliability in LLM Deep Research Agents

Key points

Evaluation approach

Limitations

Takeaways

Tags