Arbor vs EvoScientist: Two Organizational Philosophies of Autonomous Research Agents

Two Solutions to the Same Problem

Autonomous research agents face one core challenge: how to turn isolated trial-and-error into accumulating knowledge?

Arbor and EvoScientist offer two very different answers.

• Arbor (Jin et al., *Toward Generalist Autonomous Research via Hypothesis-Tree Refinement*, arXiv:2606.11926, 2026) is like a growing tree — every experiment is a node, and success or failure changes the shape of the whole tree.
• EvoScientist (Lyu et al., *Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery*, arXiv:2603.08127, 2026) is like an evolving ecosystem — three specialized agents evolve and collaborate with the help of memory banks.

The difference is not implementation detail but a fundamentally different understanding of how research should be organized.

Architecture Comparison: Tree vs. Ecosystem

Arbor: Hypothesis Tree + Coordinator/Executor

A persistent Coordinator owns the hypothesis tree (search frontier + memory + audit trail) and dispatches short-lived, sandboxed Executors that run experiments strictly bound to assigned hypotheses.

Core mechanism: Hypothesis Tree Refinement (HTR). Tree nodes are ⟨hypothesis, insight, metadata⟩. Leaves are executable experiments; internal nodes are abstract directions. Experimental evidence is backpropagated upward, influencing decisions at parent and sibling nodes.

Key constraints:

• Executors must remain hypothesis-bound — they cannot change hypotheses unilaterally
• A Merge Gate — results are adopted only after validation on held-out test data
• Strict dev/test separation to prevent overfitting the development set

EvoScientist: Three Agents + Dual Memory Banks

Three specialized agents: the Researcher Agent (RA) generates ideas, the Engineer Agent (EA) executes experiments and writes code, and the Evolution Manager Agent (EMA) distills experience. Two memory modules (Ideation Memory, Experimentation Memory) are stored in a vector database (mxbai-embed-large) with semantic retrieval.

Key constraints:

• RA must retrieve Ideation Memory when generating ideas, avoiding previously failed directions
• EA retrieves Experimentation Memory before execution, reusing effective strategies
• EMA continuously distills experience and updates the memory banks

Memory Structure: The Deepest Difference

Arbor: the tree is structured causality

The hypothesis tree is not "storage" but "reasoning." When a leaf fails, the failure isn't merely recorded — it directly changes the tree's topology: pruning a subtree, adjusting a parent's insight, or making a sibling more attractive.

The tree encodes logical relations between hypotheses:

• Parents are generalizations of children ("improve data augmentation" → "add Gaussian noise")
• Siblings are mutually exclusive competing directions
• A root-to-leaf path is a complete research narrative

Advantage: transparent reasoning. You can point at the tree and say, "We tried three directions; A and B failed for reason X, so C is more promising."

Cost: tree depth is limited (depth=2 by default) — deeper trees explode the coordinator's context.

EvoScientist: vector memory as a retrievable experience base

The memory banks are flat and unstructured: each experience is a vector retrieved by semantic similarity. This resembles how human researchers accumulate literature — you retrieve related work when facing a new problem, but conclusions aren't causally linked into a unified reasoning graph.

Advantage: cross-project reuse. A failure in one project can, via semantic retrieval, inform a related project.

Cost: logical structure between hypotheses is lost. You know "this direction failed before," but not *why* it failed or how it relates to the current attempt.

Handling Failure

Arbor: failure must reshape the tree

Failed experiments don't silently disappear. They propagate upward and may lower a parent's feasibility confidence, prune an entire subtree, or raise siblings' relative attractiveness. The cost of failure is immediate and structural — a tree won't fail twice for the same reason, because after the first failure that direction ceases to exist.

EvoScientist: failure is stored but not necessarily avoided

Failures go into the "failed directions" partition of Ideation Memory, and the RA retrieves them to avoid those directions. But retrieval is similarity-based: if a new idea superficially differs from a failed direction yet shares the same deep assumption, vector search may miss the connection. Arbor's tree cannot miss — a parent's failure directly de-prioritizes all descendants, however different they look on the surface.

Evaluation Philosophy: Performance vs. Idea Quality

Arbor: held-out performance only

Six tasks, all executable code improvement, measured by hard metrics: fewer training steps (Optimizer Design), lower validation loss (Architecture Design), higher pass rates (Terminal-Bench), higher accuracy (BrowseComp), better pass-gap (Data Synthesis), and medal rate on MLE-Bench Lite (86.36% Any Medal).

EvoScientist: idea quality matters too

Evaluation spans two dimensions: idea quality (novelty / feasibility / relevance / clarity, judged by both humans and AI) and code execution success rate. Six papers were submitted to ICAIS 2025, exceeding seven SOTA baselines across four dimensions.

This reflects a fundamental split: Arbor treats research as an optimization problem (improving an artifact under a given objective); EvoScientist treats it as a discovery process (producing new, valuable ideas).

Evolution Mechanisms

• Arbor — topological: each coordinator loop observes the frontier, expands promising leaves, dispatches executors, backpropagates evidence, and prunes/merges. The tree grows and dies; pruned subtrees don't revive.
• EvoScientist — "chemical": each loop retrieves memory, generates and executes, then distills insights into memory. The memory bank is like a continuously concentrating broth; experience "inheritance" carries across projects.

Which Is Better? It Depends

| Dimension | Arbor | EvoScientist | |---|---|---| | Best scenario | Executable engineering optimization | Creative open-ended discovery | | Memory structure | Tree (structured causality) | Vector store (semantic retrieval) | | Failure handling | Pruning (hard constraint) | Record + avoid (soft constraint) | | Cross-project reuse | Limited (trees are task-bound) | Strong (memory is cross-task) | | Explainability | High (tree topology = reasoning chain) | Medium (retrievable but implicit relations) | | Startup cost | Low (single-task loop) | High (advantage requires memory accumulation) | | Scalability | Depth-limited | Memory can grow indefinitely | | Evaluation | Held-out performance gain | Idea quality + execution success |

Choose Arbor if: you have a clear optimization goal; you need strict dev/test separation; you want an auditable research process; the task is one-off with no cross-project accumulation needed.

Choose EvoScientist if: you need genuinely novel scientific ideas; you run long-term projects requiring cross-project experience; task boundaries are too fuzzy to predefine an objective; you have the compute to sustain memory banks and embedding retrieval.

A Compelling Complementarity

The two systems don't conflict — they could be combined:

1. Use Arbor's hypothesis tree for structured within-task exploration (which directions are dead, which are alive, how evidence propagates) 2. After each task, the EMA distills the tree's insights into the vector memory bank 3. When the next task starts, the RA retrieves memory to initialize a new hypothesis tree

This yields both causal transparency within a task and evolutionary capability across tasks.

Conclusion: A Divergence of Organizational Philosophy

Arbor believes research is a structured local search — like chess, each move builds on analysis of the current position, with a "PI" watching the whole board, timely loss-cutting on failure, and independent validation on success.

EvoScientist believes research is the long-term evolution of a population — like an ecosystem, where diversity beats a single optimum, distinct "species" (agents) evolve independently, and failure is not an endpoint but part of the gene pool.

Both philosophies are sound. A truly autonomous research system may need to learn both chess and evolution.

References

• Jin, J. et al. (2026). Toward Generalist Autonomous Research via Hypothesis-Tree Refinement. arXiv:2606.11926.
• Lyu, Y. et al. (2026). Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery. arXiv:2603.08127.