The Awakening of AI Scientists: When Machines Begin to Evolve Themselves

Key points

The post (written in a deliberately classical Chinese style) reviews EvoScientist, an open-source evolving multi-agent AI scientist framework, and its official skill repository EvoSkills, which offers 15 professional skill packages.

Three generations of AI scientist systems

• Generation 1 (2020–2023), single-agent: AutoGPT, BabyAGI — one LLM instance does everything; lacks specialization and loses direction on complex tasks.
• Generation 2 (2023–2025), multi-agent: MetaGPT, ChatDev, AutoGen — role specialization (PM, architect, engineer, tester) with document-based collaboration, but static workflows and no learning from history.
• Generation 3 (2025–), evolving multi-agent: EvoScientist, GNoME, FunSearch — persistent memory plus self-evolution.

EvoScientist architecture: three cooperating agents

• Researcher Agent (RA): generates research ideas via memory loading from idea memory (M_I), literature anchoring with paper-navigator (30–50 papers, building a Challenge-Insight Tree), multi-role ideation (innovator / pragmatist / critic), iterative refinement, ELO tournament ranking (initial 1500, K=32, pairwise comparison to pick Top 3), and proposal expansion.
• Engineer Agent (EA): a four-stage gated pipeline with attempt budgets — initial implementation (≤20 attempts), hyperparameter tuning (≤12), proposed method (≤12, significance p < 0.05), ablation studies (≤18). Gate conditions prevent wasting compute on failed stages.
• Evolution Manager Agent (EMA): runs three evolution mechanisms — IDE (extract promising directions into M_I), ESE (extract reusable experiment strategies into experiment memory M_E), and IVE (classify failures as implementation vs. fundamental-direction failures, feeding the latter back into M_I).

EvoSkills: 15 skills in 4 categories

• Research ideation: research-ideation (idea generation with counter-intuitive rules like prioritizing failure scenarios over incremental improvements), research-survey (adaptive outlines, drafting-then-expanding), paper-navigator (seven discovery paths, TLDR/citation/code-availability assessment).
• Experiment execution: experiment-pipeline, experiment-craft (five-step debugging, change one variable at a time), experiment-iterative-coder (plan → code → lint/pytest → score loop).
• Paper writing: paper-planning (reverse story design, write the rejection letter first), paper-writing (11-step workflow, topic sentences first), paper-review (rejection simulation), paper-rebuttal (color-coded reviewer comments; submit rebuttals even against extreme scores), paper-figures (spec-driven figure-spec.md before rendering matplotlib), academic-slides (claim-style titles, one idea per slide).
• Self-evolution: evo-memory (persistent memory retrieved via embedding cosine similarity; IDE/IVE/ESE mechanisms), nano-banana (Gemini-based slide/illustration generation with browser review), evomath-tao (Terence Tao–style proof workflow with honest status labels: PROVED, REFUTED, VERIFIED_NUMERICALLY, CONJECTURED, HANDED_OFF).

Experimental validation

• Idea generation: EvoScientist beat 7 SOTA systems (including AI Scientist-v2, AI-Researcher, InternAgent) on all four automatic dimensions (novelty, feasibility, relevance, clarity); blind human evaluation by 3 PhD experts confirmed significant advantages in novelty and feasibility, attributed mainly to the memory mechanism.
• End-to-end execution: code execution success rate rose from 42% without memory to 78% with memory (+36 points), largely thanks to M_E storing previously successful code strategies.
• The author suggests additional ablations (removing M_I, M_E, or both) to quantify each mechanism's contribution.

Limitations and future directions

• Limitations: memory contamination risk (mitigations: periodic memory audits, confidence scores), unverified long-term evolution stability (catastrophic forgetting over 50+ cycles), and unverified cross-domain generalization beyond computer science.
• Six proposed research directions: hierarchical memory architectures (inspired by Neural Turing Machines / DNC), multi-objective ELO optimization (e.g., NSGA-II), DAG-based automated skill orchestration, cross-domain transfer of experiment strategies, multi-discipline case studies (chemistry, biology, physics), and human-AI hybrid intelligence collaboration protocols.

Conclusion

The author concludes that EvoSkills' core contributions are its systematic skill taxonomy (4 categories, 15 skills), the deep deconstruction of the evo-memory self-evolution mechanism, and the identification of 6 knowledge gaps — while noting that true autonomous AI scientists remain some distance away.

References cited in the post

1. Lyu, Y., et al. (2026). *EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery*. arXiv:2603.08127. 2. *EvoSkills: Official Skill Repository for EvoScientist*. https://github.com/EvoScientist/EvoSkills 3. Graves, A., Wayne, G., & Danihelka, I. (2014). Neural Turing Machines. arXiv:1410.5401. 4. Graves, A., et al. (2016). Hybrid computing using a neural network with dynamic external memory. *Nature*, 538(7626), 471-476. 5. Lewis, P., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. *NeurIPS*, 33, 9459-9474.