327 Real PMIDs, Only 3 Correct Doors: When Biomedical Agent Citations Exist but Don't Point Anywhere

Forum topic · 小凯 · 2026-10-22

Summary

A review of the paper "Real PMIDs, Wrong Papers: Harness Design for Biomedical Claim-Verification Agents" (arXiv: 2610.29812). The study tests Gemini Flash-lite backbones on 100 SciFact claims across four harnesses: closed-book, single-shot RAG, ReAct agent, and an oracle with gold evidence. Single-shot keyword retrieval recalled only 6% of gold papers due to PubMed's AND semantics; letting the model rewrite short entity-focused queries tripled recall (19-20%, accuracy 62.0). The oracle reached 79-86% accuracy, showing retrieval—not reasoning—is the bottleneck. Most strikingly, an audit of closed-book citations found 98% of the 335 PMIDs genuinely exist in PubMed, yet only 3 of 327 existing citations (0.9%) were judged relevant to the claim, with zero hitting gold-standard papers. By contrast, retrieval-grounded agent citations were 66-70% relevant. The paper argues existence checks (resolvable PMIDs/DOIs) are a weak safeguard: citations that exist but are irrelevant pass all validation. Key engineering rule: only allow citations drawn from actual tool-returned retrieval results. This extends Walters & Wilder (2023) from fabricated citations to real-but-wrong ones, a harder-to-detect failure mode for scientific AI agents.

Overview

This post reviews *Real PMIDs, Wrong Papers: Harness Design for Biomedical Claim-Verification Agents* (Saqlain, 2026; arXiv: 2610.29812), the first piece in a three-part series on the "texture of evidence." The central thesis: a citation is a pointer, and existence is not relevance. An agent's citation can pass every existence check—valid PMID, real paper—while pointing to a paper that has nothing to do with the claim.

Key points

Experimental setup: N=100 claims sampled (seed 0) from the SciFact dev set (SUPPORT / CONTRADICT / NOT ENOUGH INFO labels), fixed backbone models (gemini-3.1-flash-lite and gemini-3.5-flash-lite), fixed prompts and JSON output ({label, cited_pmids, rationale}), temperature 0. Only the harness varies: closed-book, single-shot RAG, ReAct agent (max 4 PubMed tool calls), and an oracle fed gold abstracts. All tool calls and outputs are cached and published for offline reproduction.
Single-shot RAG fails badly: feeding the full claim as a PubMed keyword query yields 6% gold recall (4/64 evidence-bearing claims). PubMed's AND semantics mean each query term narrows the result set. Worse, retrieval did not convert overconfident errors into correct answers—it converted them into abstentions: 59% and 70% of predictions were NEI vs. 36% in gold, with no accuracy gain (53.0 vs 55.0, p=0.89; 49.0 vs 59.0, p=0.24).
Query rewriting is the model's biggest contribution: letting the agent write short, entity-centric queries ("chenodeoxycholic acid energy expenditure") tripled recall to 19-20% and lifted accuracy to 62.0 on both backbones—though none of these small-sample differences were statistically significant. The agent even silently fixed the misspelled term "Chenodeosycholic" in the claim text.
The oracle shows retrieval is the bottleneck: with gold abstracts provided, accuracy jumps to 86.0% and 79.0% (p<10⁻⁴ and p=0.006 vs the agent)—24 and 17 points above the best realistic harness—and excessive abstention disappears. Reasoning isn't the weak link; retrieval is.
Behavioral detail: agents used only 1.42/1.70 tool calls on average; in evidence-bearing claims where they never saw the gold abstract, they still answered Support/Contradict correctly 75% of the time—but recall on genuinely NEI claims dropped to 33-42%.
The headline audit: every closed-book citation was verified against live PubMed, then a cross-family judge (calibrated on 50 matched/mismatched controls) rated relevance:
Existence: 327/335 PMIDs real (98%)
Gold hits: 0 of 327
Relevance: 3 of 327 (0.9%)
Absurd examples included citing "The proper eyeglasses for children" for a TNF-α/IL-1 cytokine-regulation claim.
Grounded retrieval fixes it: the same judge rated agent citations 66-70% relevant (141/215 and 128/184), with non-retrieved citation rates of just 0-2.6%. The gap from 0.9% to ~70% is not model size—it is whether the model saw the paper.
Existence checks are a decoy: resolvable PMIDs, working DOIs, valid formats—327 citations passed all of it with zero alarms. This advances Walters & Wilder (2023), who studied *fabricated* citations; here the failure mode is *real but irrelevant* citations, which survive every standard compliance check.

Takeaways

Citations generated without retrieval should not be shown to users, even if IDs resolve.
Engineering iron rule: only permit citations drawn from actual tool-returned results.
Spend on iterative retrieval and citation chasing before buying stronger backbones or fancier prompts; enforce a minimum evidence threshold before answering to avoid confidently-wrong judgments.

Limitations (as stated by the author)

N=100 with wide confidence intervals; no significant differences among realistic harnesses; single-vendor small backbones; gold-label matching via title normalization; a single LLM judge validated on only 50 controls. This is a pilot, not a final verdict—but the 0.9% relevance rate and the oracle gap are structural findings that these limitations do not shake.

References

Saqlain, M. (2026). *Real PMIDs, Wrong Papers*. arXiv: 2610.29812. https://arxiv.org/abs/2610.29812 · Code: https://github.com/saqlain2204/real-pmids-wrong-papers
Wadden, D., et al. (2020). Fact or Fiction: Verifying Scientific Claims. EMNLP 2020. (SciFact)
Yao, S., et al. (2023). ReAct. ICLR 2023.
Lewis, P., et al. (2020). Retrieval-Augmented Generation. NeurIPS 33.
Walters, W.H., & Wilder, E.I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. *Scientific Reports* 13, 14045.
Skarlinski, M.L., et al. (2024). Language Agents Achieve Superhuman Synthesis of Scientific Knowledge. arXiv: 2409.13740. (PaperQA2)