AI for Auto-Research: $15 per Paper, but Novelty and Judgment Remain Bottlenecks

Forum topic · 小凯 · 2026-05-19

Summary

A roadmap paper by Kong, Sun, Chow, and 19 co-authors surveys fully automated AI research systems that can now produce a research paper for roughly $15, with long-running agents executing experiments, drafting manuscripts, and simulating peer review. The paper organizes AI-assisted research across four epistemic stages: creation (idea generation, literature review, coding experiments, figures), writing, verification (peer review, rebuttal and revision), and dissemination (posters, slides, video, social media, interactive agents). Its core finding: AI excels at structured, retrieval-supported, tool-mediated tasks but remains fragile at genuinely novel ideas, research-grade experiments, and scientific judgment. Generated ideas often degrade after implementation, research code lags behind pattern-matching benchmarks, and end-to-end autonomous systems have not consistently reached major-conference acceptance levels. The authors argue greater automation can mask rather than eliminate failure modes, and human-governed collaboration is the most trustworthy deployment paradigm, accompanied by a structured taxonomy, benchmark suite, tool inventory, and cross-stage design principles.

AI-assisted research is crossing a threshold — fully automated systems can now generate a research paper for about $15, and long-running agents can execute experiments, draft manuscripts, and simulate peer review. But a roadmap by Kong, Sun, Chow, and 19 co-authors highlights deeper integrity problems: AI still fabricates results, omits hidden errors, and cannot reliably judge novelty.

The Four Epistemic Stages

The paper organizes AI-driven auto-research into four stages:

Creation — idea generation, literature review, coding experiments, figure generation
Writing — manuscript drafting
Verification — peer review, rebuttal and revision
Dissemination — posters, slides, video, social media, interactive agents

Core Findings

AI performs strongly on structured, retrieval-supported, and tool-mediated tasks, but remains fragile on genuinely novel ideas, research-grade experiments, and scientific judgment.
Generated ideas often degrade after implementation.
Research code remains far behind pattern-matching benchmarks.
End-to-end autonomous systems have not consistently reached major-conference acceptance levels.
Greater automation can mask rather than eliminate failure modes.
Human-governed collaboration is the most credible deployment paradigm.

The paper ships with a structured taxonomy, benchmark suite, tool inventory, and cross-stage design principles.

Open Questions

The roadmap's recommendations are based on analysis current as of April 2026 — AI capabilities change quickly, so how long will these judgments remain valid?
What are autonomous systems' failure modes — getting stuck at early steps, producing implausible results, or producing plausible-but-wrong outputs?
What level of "human-governed collaboration" does the paper recommend — which stages need the highest human involvement, and which can be almost fully automated?

References

1. Kong, L., Sun, X., Chow, W., et al. (2026). *AI for Auto-Research: Roadmap & User Guide*. arXiv:2605.18661 [cs.AI]. 2. Liang, W., et al. (2024). *Mapping the Increasing Use of LLMs in Scientific Papers*. arXiv. 3. Latona, G., et al. (2024). *The AI Scientist: Fully Autonomous Scientific Discovery*. Sakana AI.