Cited but Not Verified: The AI You Trust With Citations Is Misleading You
> Last month I used Perplexity to look up a medical statistic. It produced a report with 8 citations. The links were clickable, the pages existed, and the content matched the topic. But when I clicked through to the third citation to verify, the original text never mentioned the number Perplexity claimed. It had attributed data from a different article to this citation. > > I thought it was a one-off. Then I read this paper — and realized this isn't a bug, it's a feature.
The paper comes from PwC's Commercial Technology and Innovation team, titled Cited but Not Verified. Six authors (Hailey Onweller, Elias Lumer, et al.) did something brutal: they built an automated framework that mass-scraped deep research reports from 14 mainstream LLMs, then verified every citation for link validity, content relevance, and factual accuracy.
The results are chilling.
---
1. Surface Polish vs. Substance: A Large-Scale Magic Trick
Core results table:
| Dimension | Frontier model performance | What it means | |:---------|:-----------:|:---------| | LinkWorks (link valid) | >94% | Links you click almost always open | | Relevant Content | >80% | The page really is on topic | | Fact Check (factually accurate) | 39–77% | But the source doesn't necessarily support the AI's claim |
> These three dimensions form an increasing verification-difficulty pyramid: > - LinkWorks is the base: does the URL return HTTP 200? No content understanding needed. > - Relevant Content is the middle: is the page on the same topic? Requires semantic understanding, not sentence-level checking. > - Fact Check is the top: is the specific fact (number, date, claim) supported by the source? Requires fine-grained fact-checking. > > The paper's key finding: the lower-level metrics look great, the most critical one looks bad.
Notice the cliff: near-perfect link validity, strong relevance — then factual accuracy gets cut roughly in half.
What this means:
> The AI hands you a "professional-looking" research report. Citations, links, sources. You click a few links, pages exist, content is relevant. You relax. > > What you didn't do: check every specific number the AI claimed against the original text.
And that's exactly where the problem lives.
---
2. The Mirror for 14 Models
The paper evaluated 14 models across OpenAI, Anthropic, Google, and open source:
| Model | Task success | LinkWorks | Relevant Content | Fact Check | |:-----|:----------:|:---------:|:----------------:|:--------------:| | Claude Opus 4.5 | 90% | 98.7% | 95.7% | 76.8% | | Claude Haiku 4.5 | 83% | 98.9% | 91.1% | 68.9% | | GPT-5.2 | 100% | 98.3% | 92.3% | 58.8% | | Claude Sonnet 4.6 | 93% | 99.2% | 89.8% | 58.7% | | Claude Opus 4.6 | 93% | 97.2% | 83.9% | 54.2% | | Codex | 100% | 96.9% | 91.9% | 54.1% | | Claude Sonnet 4.5 | 97% | 98.9% | 88.3% | 51.8% | | Gemini 3.1 Pro | 90% | 94.1% | 80.7% | 48.5% | | GPT-5.4 | 100% | 100% | 93.7% | 47.7% | | Gemini 3 Flash | 100% | 94.7% | 82.9% | 45.2% | | GPT-5 Mini | 100% | 99.3% | 87.4% | 38.9% | | Pixtral Large | 17% | 100% | 64.9% | 51.4% | | Llama 4 Maverick | 30% | 80.8% | 60.6% | 34.3% | | OSS-120B | 40% | 83.9% | 68.7% | 24.4% |
> Data source: paper Table 1, aggregated over 130 research queries.
Several painful takeaways:
First, Anthropic wins on "telling the truth." Claude Opus 4.5 hits 76.8% Fact Check, the highest of all models. Though its task success rate is "only" 90%, its citations are the most reliable. The authors' analysis: selective citation may beat exhaustive citation — cite fewer, but ensure accuracy.
Second, OpenAI's "100% task success" is a beautiful trap. GPT-5.4, GPT-5.2, Codex, and GPT-5 Mini always produce cited reports, but their Fact Check scores are only 38–59%. They never refuse to generate a report — but nearly half the factual citations in those reports are problematic.
Third, open-source models can't even reliably generate reports. Llama 4 Maverick manages only 30% task success; Pixtral Large just 17%. They either skip citations or produce formats too messy to parse.
Fourth, GPT-5.4 scores lower on Fact Check than GPT-5.2 (47.7% vs 58.8%). A bigger model with more tool calls actually got less factually accurate — leading to the paper's most counterintuitive finding:
---
3. The More You Search, the More You Get Wrong
The authors ran an ablation: controlling GPT-5.4 and Claude Opus 4.6, they scaled tool calls from 2 to 150 and tracked citation quality.
| Tool calls | GPT-5.4 FactCheck | Claude Opus 4.6 FactCheck | |:-----------:|:-----------------:|:-------------------------:| | 2 | 78.6% | 80.0% | | 10 | 45.9% | 74.4% | | 30 | 43.0% | 69.2% | | 50 | 38.0% | 61.2% | | 70 | 35.5% | 61.7% | | 100 | 37.2% | 58.7% | | 150 | 16.7% | 57.9% |
> Data source: paper Tables 2 and 3.
Look at the GPT-5.4 line: from 78.6% at 2 calls to 16.7% at 150 — ask it to "search a bit more" and its citation accuracy goes from passable to almost all wrong. Claude Opus 4.6 is more stable but still falls from 80% to 58%.
> The paper attributes this to information overload. Synthesizing from more sources makes three error types more likely: > - Conflation: mixing source A's number with source B's claim into a "new fact" > - Misattribution: attributing a fact supported by source A to source B > - Over-inference: "reading out" conclusions the source doesn't explicitly support > > Crucially, LinkWorks and Relevant Content stay nearly unaffected as search depth grows (always >92%); only Fact Check deteriorates. The model isn't failing to find good sources — it's failing to synthesize facts accurately from many sources.
A scary scenario:
> You query an important topic. The AI returns a report citing 50 sources. You click a few — they open, content is relevant. Seems reliable. > > But if that report came from GPT-5.4, only 16.7% of its cited facts might survive verification. > > What you're seeing isn't "deep research" — it's wrapped deep hallucination.
---
4. Why "Looks Correct" Is More Dangerous Than "Is Correct"
The paper's ethics statement, quoted directly:
> *"High link validity and content relevance scores may create a false sense of trust."*
That's the core. The most dangerous hallucination isn't obvious nonsense — it's nonsense that looks completely plausible.
Your guard drops to zero when an AI report has:
- ✅ Inline citations
- ✅ Working links
- ✅ Pages that exist
- ✅ Relevant content
- ✅ Well-formatted citations
- ✅ Authoritative-looking sources
> Previous research (Yuan et al., 2026) classifies citation hallucination levels: > - Fully fabricated: link doesn't exist (easiest to spot) > - Dead link: page once existed but was removed > - Misattribution: link works, but the claimed fact comes from another article > - Over-interpretation: link works and is relevant, but the AI "read out" unsupported content > > This paper's framework mainly exposes the last two — where the surface is fine but the substance is rotten.
---
5. The Framework Itself Has Problems
The authors are honest about limitations. The most critical:
LLM-as-a-judge evaluation may itself be biased.
Fact Check and Relevant Content are scored by another LLM. If that evaluator also hallucinates, it may score citations wrongly. The authors calibrated with human judgments (50–100 samples per model), but that may be too few for a fine-grained fact-checking task.
Also, web content changes over time. Links that work today may 404 tomorrow; page content may be edited. Re-running later could shift the numbers.
And my own observation: the paper doesn't distinguish "factual error" from "interpretive difference." If a source says "US GDP grew 2.1%" and the report says "about 2%", is that an error? The framework's binary support/not-supported judgment may obscure gray areas.
---
6. What This Means for You
My judgment: "cited AI reports" are becoming the most refined information trap of the 21st century. The model doesn't intend to deceive — but it systematically produces content that looks credible yet fails verification. Because surface metrics (links, relevance) look so good, users almost never discover these problems spontaneously.
A restaurant analogy: every dish's menu lists sourcing — A5 wagyu from Japan, truffles from Alba, caviar from the Caspian. All specific, all real. But the "wagyu" on your plate is Australian beef. The label is real; the label and the dish don't match. Where the analogy breaks down: scale. A restaurant deceives a few people; a deep research AI generates millions of reports daily, each with dozens of citations. At 50% problematic factual citations, that's systematic information pollution.
Practical advice from the paper:
| Role | What to do | |:-----|:-----------| | Everyday users | Don't relax because the AI gave citations. Randomly spot-check 2–3 | | System designers | Integrate citation-quality monitoring into agent pipelines, not just surface metrics | | Researchers | Keep building automated fact-verification frameworks, especially for multi-source synthesis |
---
7. Closing Thoughts
Reading this paper, one question kept nagging me: when we use AI for research, what exactly are we trusting?
If citations score near-perfect on "link valid" and "content relevant" but only 40–70% on "factually accurate," then we're not trusting "the AI found correct information for me" — we're trusting "the AI made me feel it found correct information."
That distinction is the line between trust and information manipulation.
What PwC's authors did matters: they turned invisible hallucination into visible data. Before them, we knew AI hallucinates, but citation-level hallucination was hard to measure systematically. Now there's a framework — an evaluation pipeline that can run on any deep research AI.
But more important than the framework is an attitude: staying skeptical of AI citations isn't distrusting technology — it's respecting facts.
Next time you see an AI-generated research report with neatly formatted, gleaming citations —
Ask yourself: did I just open a doorway to truth, or the wrapping paper of a hallucination?
---
Paper Details
| Item | Content | |:-----|:--------| | Title | Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents | | Authors | Hailey Onweller, Elias Lumer, Austin Huber, Pia Ramchandani, Vamse Kumar Subbiah, Corey Feld | | Institution | PricewaterhouseCoopers (PwC), U.S. Commercial Technology and Innovation Office | | arXiv ID | 2605.06635 | | Published | 2026-05-07 | | Category | cs.CL (Computation and Language) | | Core claim | Serious gap between surface citation quality (link validity, relevance) and factual accuracy in LLM deep research agents; 14 frontier models achieve only 39–77% factual accuracy, which worsens with deeper search | | Framework | Three-stage pipeline: Markdown AST parser extracts citations → URL fetching → three-dimension evaluation (LinkWorks / Relevant Content / Fact Check) | | Models evaluated | 14 LLMs: OpenAI (GPT-5.4/5.2/5 Mini/Codex), Anthropic (Claude Opus/Sonnet/Haiku 4.5/4.6), Google (Gemini 3.1 Pro/3 Flash), open source (Llama 4 Maverick/Pixtral Large/OSS-120B) | | Data | 130 research queries from DeepResearchBench and BrowseComp | | Key results | Best models: >94% links, >80% relevance, but only 39–77% facts; GPT-5.4 FactCheck drops from 79% to 17% between 2 and 150 tool calls | | Paper link | https://arxiv.org/abs/2605.06635 |