EvoScientist: Huawei's Self-Evolving AI Scientist That Learns Research Intuition From Experience

Forum topic · 小凯 · 2026-03-20

Summary

EvoScientist is a multi-agent AI scientist system developed by Huawei (arXiv:2603.08127) designed to address a key limitation of existing AI research systems like Sakana's The AI Scientist and Google's AI Co-Scientist: their lack of persistent memory. Most AI scientists start from scratch on every task, repeating past mistakes. EvoScientist introduces three specialist agents — a Researcher Agent (idea generation, literature review, tree search, Elo tournament selection), an Engineer Agent (code generation across four experiment phases), and an Evolution Manager Agent (EMA) that distills reusable knowledge after each task into two persistent memory stores: an Ideation Memory of promising and failed research directions, and an Experimentation Memory of effective code strategies. Three evolution mechanisms — idea direction, idea validation, and experiment strategy evolution — enable cross-task learning. Benchmarks show EvoScientist outperforming seven baseline systems by 29–93% average win margins, and code execution success rates improving from 34.39% to 44.56% after evolution. Most notably, six fully autonomous papers submitted to ICAIS 2025's AI Scientist Track were all accepted (100% vs. the conference's 31.71% overall rate), with one winning Best Paper Award. The project is open-sourced on GitHub, though limitations remain: low success rates on implementing novel methods (21.57%), dependence on external APIs, and validation limited to machine learning tasks.

EvoScientist: A Self-Evolving AI Scientist That Accumulates "Research Intuition"

Paper: EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery Code: github.com/EvoScientist/EvoScientist Authors: Huawei team (Yougang Lyu, Xi Zhang, Xinhao Yi, et al.)

The Problem: AI Scientists Have "Goldfish Memory"

Current AI scientist systems can generate papers end-to-end (Sakana's The AI Scientist) or propose hypotheses (Google's AI Co-Scientist), but they are stateless: every run starts from zero. They don't remember why a previous experiment failed, which activation function caused gradient explosions, or which datasets need preprocessing first. The result is repeated mistakes and wasted compute on dead-end directions.

Human researchers are efficient because of accumulated research intuition — the ability to quickly judge whether an idea is viable, built from years of trial and error. EvoScientist aims to give AI scientists exactly this capability: it evolves, getting smarter with use.

Architecture: Three Agents + Two Notebooks

EvoScientist combines three specialist agents with two persistent memory stores:

Agents

Researcher Agent (RA) — the "creative director". Given a research goal, it performs literature search via the Semantic Scholar API, generates candidate ideas informed by Ideation Memory, refines them with tree search (propose–review–improve), ranks them via Elo-based pairwise tournaments, and expands the winner into a full research proposal.
Engineer Agent (EA) — the "builder". It retrieves successful code strategies from Experimentation Memory, then performs iterative code tree search across four experiment phases: (1) initial implementation, (2) hyperparameter tuning, (3) implementing the proposed method, (4) ablation studies. Code runs in a sandbox with logs and diagnostics.
Evolution Manager Agent (EMA) — the differentiator. After each task, EMA reviews the entire interaction history, distills reusable knowledge (viable directions, failure lessons, effective code strategies), and writes it back into the two memories.

Memories

Ideation Memory: viable research directions (from top-scored ideas) and failed approaches, so RA avoids repeating dead ends.
Experimentation Memory: data preprocessing strategies, training tricks (e.g., when AdamW beats SGD), and debugging patterns, so EA implements faster.

Three Evolution Mechanisms

1. Idea Direction Evolution: EMA takes the top-3 Elo-rated ideas per task and LLM-summarizes their commonalities into reusable direction guidance. 2. Idea Validation Evolution: failed implementations and underperforming experiments are analyzed and recorded as lessons, preventing RA from re-proposing similar ideas. 3. Experiment Strategy Evolution: EMA mines EA's full search trajectory (successes and failures) for effective preprocessing pipelines, hyperparameter configs, and error diagnosis patterns.

Memory retrieval uses mxbai-embed-large embeddings with cosine similarity; updates are LLM-generated summaries written to the respective store.

Results

Idea Quality

Against 7 baselines, EvoScientist's average win-rate advantages (Gemini-3-flash judge):

| Baseline | Type | Avg. win advantage | |---|---|---| | Virtual Scientist | open-source | +29.17% | | AI-Researcher | open-source | +56.25% | | InternAgent | open-source | +93.34% | | AI Scientist-v2 | open-source | +45.83% | | Hypogenic | commercial | +46.00% | | Novix | commercial | +80.83% | | K-Dense | commercial | +69.17% |

Human evaluation by three PhD-level experts: novelty 82.5% win rate, feasibility 72.5%, relevance 62.5%, clarity 75%.

Code Execution Success

| Phase | Before evolution | After evolution | |---|---|---| | Initial implementation | 54.88% | 68.29% | | Hyperparameter tuning | 36.59% | 46.34% | | Proposed method | 20.33% | 21.57% | | Ablation | 25.20% | 42.02% | | Average | 34.39% | 44.56% |

End-to-End: 6 Papers Accepted at ICAIS 2025

Six fully autonomous papers submitted to ICAIS 2025's AI Scientist Track were all accepted (100%, vs. 31.71% overall conference rate), with one winning Best Paper Award and one the AI Reviewer's Appraisal Award. Reviewers praised novelty and clear motivation (100%) and reasonable experiment design (83%); common criticisms were incomplete ablations (50%) and insufficient related-work discussion (33%).

Comparison with The AI Scientist

| Dimension | The AI Scientist (Sakana) | EvoScientist (Huawei) | |---|---|---| | Core innovation | First end-to-end automation | First self-evolving system | | Memory | None persistent | Two continuously updated stores | | Learning | Single run, tree search | Cross-task evolution | | Code generation | Templates (v1) | Autonomously retrieved strategies |

The AI Scientist's agentic tree search happens within a task and is discarded afterward; EvoScientist's EMA preserves distilled knowledge across tasks, like a scientist consulting a research log before a new project.

Limitations

1. Success rates remain low for implementing novel methods (21.57% even after evolution). 2. External API dependence: Semantic Scholar, multiple LLM providers, Ollama — costs and reliability tied to third parties. 3. Domain limits: validated mainly in machine learning; physical sciences would require major architectural changes.

Key Insight

> "The bottleneck for AI scientists is not model capability, but memory mechanisms."

EvoScientist demonstrates that even state-of-the-art LLMs stagnate without a way to accumulate and reuse experience. True intelligence — human or machine — requires learning from the past. Its two "research notebooks" give AI scientists something resembling human research intuition: knowing which directions are promising, which methods work, and which pitfalls to avoid.

References: 1. Lyu et al., "EvoScientist", arXiv:2603.08127, 2026 — https://arxiv.org/abs/2603.08127 2. https://github.com/EvoScientist/EvoScientist 3. Lu et al., "The AI Scientist", arXiv:2408.06292 4. Yamada et al., "The AI Scientist-v2", 2025 5. Gottweis et al., "Towards an AI Co-Scientist", arXiv:2502.18864 6. Beel et al., "An Evaluation of Sakana's AI Scientist", arXiv:2502.14297 7. https://deepresearch-bench.github.io/