Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents

• Source: arXiv:2503.24047 (March 2025)
• Authors: Shuo Ren, Can Xie, Pu Jian, Zhenjiang Ren, Chunlin Leng, Jiajun Zhang
• Type: Survey
• Category: Deep Research

Overview

This survey systematizes the emerging field of LLM-based scientific agents: autonomous systems powered by large language models that execute scientific workflows such as literature analysis, hypothesis generation, experiment design, and paper writing. It introduces a task-oriented taxonomy to organize existing work and discusses the path from current research prototypes toward genuine scientific intelligence.

Key points

• Taxonomy by research task: Agent capabilities are organized along the scientific pipeline—literature search and review, idea generation, experiment planning and execution (including code generation), data analysis, and scientific writing.
• Core building blocks: The survey decomposes agent architectures into planning, tool use, memory, self-reflection, and multi-agent collaboration, and compares how representative systems implement each component.
• Benchmarks and evaluation: It catalogs datasets and benchmarks used to assess scientific agents, and highlights gaps between offline metrics and real research quality, including issues of reproducibility and LLM-as-judge bias.
• From agents to scientific intelligence: The authors argue that scaling from task-specific assistants to end-to-end autonomous research systems requires tighter integration with scientific tools, databases, and physical lab infrastructure.

Open challenges

• Reliability and hallucination in scientific claims; need for verifiable outputs and citations.
• Insufficient unified benchmarks; private data and non-reproducible experiments.
• Cost, latency, and safety constraints of agentic systems operating in open environments.
• Multimodal and cross-lingual extension beyond English-centric corpora.

Timeline / related context

• 2019–2021: neural retrieval foundations (BERT rerankers, DPR)
• 2022–2023: RAG and retrieval–generation fusion
• 2024+: conversational/agentic search, Gen-RecSys
• 2025+: RL-trained search agents, Deep Research, and GraphRAG as growth areas

Related entries

• A Comprehensive Survey of Deep Research
• A Survey of LLM-based Deep Search Agents
• A Survey of Scientific Large Language Models
• Agentic Reasoning

Takeaways

1. Researchers: Reproduce benchmark comparisons; check statistical significance and compute cost reporting. 2. Engineers: Treat agent modules (planner, retriever, code generator) as pluggable components and assess integration cost with existing stacks. 3. Product teams: Focus on user-perceivable gains—answer trustworthiness, latency, multi-turn consistency—rather than offline metrics alone.

References

• Original paper: Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents, arXiv, March 2025.