Feynman-style essay: Why GenAI retrieval still cites retracted papers

Forum topic · 小凯 · 2026-05-03

Summary

A Chinese tech forum post reviews the GenAI Retrieval Reliability Assessment (2026.05) and argues that mainstream large language models fail at reliably excluding retracted scientific literature. The author explains that even top models like GPT-5 and Claude Opus could not fully filter retracted papers when instructed to retrieve only valid citations. The root cause, per the post, is architectural: LLMs are statistical models pretrained on data that predates many retraction notices, and typical RAG pipelines optimize for semantic relevance rather than checking authoritative, continuously updated retraction databases. The author frames science as requiring falsifiable, rigorously verified evidence rather than plausible-sounding citations, concluding that GenAI is a strong summarizer but a poor factual auditor. The practical recommendation is to build third-party cross-verification protocols—graph-based, model-independent retraction checks—especially in research, legal, and medical applications, otherwise AI-generated reports risk becoming polished but unreliable outputs.

Feynman-style essay: Do you want a clean library, or a landfill to pan for gold?

After reading the brutal GenAI Retrieval Reliability Assessment (2026.05) report, the author feels we are entrusting the future of research to "chefs who never check expiration dates."

To explain why mainstream models (ChatGPT, Claude, etc.) fail at literature retrieval, the post focuses on paper retractions.

1. The current state: AI assistants racing blindly through the paper ocean

Researchers increasingly ask AI to find literature: "Find me the latest research on gene X." The AI instantly lists 10 highly professional-looking papers — but some may have been retracted from Nature last year for data fabrication. This is the silent propagation of contaminated data.

2. The test verdict: no model avoids 100% of the garbage

The assessment gave 9 mainstream LLMs a strict instruction: retrieve literature with absolutely no retracted papers.

Even the strongest models (GPT-5, Opus) could not fully identify and exclude retracted "poison."
Why? LLMs are statistical hybrids that swallowed the whole internet during pretraining — before many retraction notices existed. Their RAG systems typically match semantic relevance only, without connecting to a real-time, authoritative "academic blacklist database." It's like a chef judging ingredients purely by looks, never checking health-authority recall lists.

3. The Feynman-style verdict: science demands falsifiable rigor

Scientific work is not about who cites the most papers — every brick must survive strict physical/logical cross-validation.

The assessment's takeaway: AI is a top-tier summarizer but currently a terrible fact auditor. Without a retraction-blocking mechanism that is independent of the model and based on hard graph-based checks, every glossy AI report is a building on quicksand.

Key takeaways

Don't treat AI output as truth in research, legal, or medical contexts.
Build a third-party cross-verification protocol.
A system that optimizes only for "relevance" without hard checks on "truth labels" is just a tireless machine for producing academic garbage.