AutoSci vs EvoScientist: A Systematic Architecture and Implementation Comparison

Forum topic · ✨步子哥 · 2026-06-01

Summary

This forum post presents a detailed technical comparison of two open-source AI research agent systems: AutoSci (Peking University DAIR Lab, arXiv:2605.31468, MIT license) and EvoScientist (arXiv:2603.08127, Apache-2.0). AutoSci is a memory-centric skill pack for Claude Code built around OmegaWiki, a file-based knowledge graph (Markdown + YAML frontmatter + JSONL edges) covering the full research lifecycle with a human-in-the-loop workflow. EvoScientist is a standalone, self-evolving agent runtime built on deepagents and LangGraph, featuring six declaratively configured sub-agents, a three-tier memory system with automatic extraction, SQLite checkpointing, multi-provider LLM routing (15+ vendors), omnichannel access (Telegram, Slack, Discord, WeChat, Feishu, and more), a multi-layer sandbox with human-in-the-loop approvals, and enterprise-grade engineering practices including ~890 tests, Ruff linting, and Docker deployment. The analysis covers design philosophy, architecture layers, runtime mechanisms, memory systems, interaction layers, security models, engineering practices, deployment topologies, and extension ecosystems, concluding that AutoSci favors transparency and human auditability while EvoScientist favors autonomy, scalability, and operational robustness.

This post compares two open-source AI research agent systems across design philosophy, architecture, memory, runtime, interaction, security, engineering, deployment, and extensibility.

Compared systems

| Dimension | AutoSci | EvoScientist | |------|---------|--------------| | Positioning | Memory-centric agent system for the full research lifecycle | Self-evolving AI scientists for end-to-end discovery | | Paradigm | Human-in-the-Loop | Human-on-the-Loop | | Report | arXiv:2605.31468 | arXiv:2603.08127 (ICAIS 2025 Best Paper) | | Team | Peking University DAIR Lab (Weitong Qian et al.) | International team (Xi Zhang, Yougang Lyu, Dinos Papakostas et al.) | | License | MIT | Apache-2.0 | | Python | >=3.9 | >=3.11 (Ruff py311) | | Entry point | Claude Code skill set (/command) | Standalone CLI/TUI (EvoSci / evoscientist) |

Key points

Philosophy: AutoSci positions itself as a "research skill extension pack for Claude Code" (30+ /command skills covering ingestion, ideation, experiments, paper writing, rebuttals). EvoScientist is a self-contained multi-agent operating system with 10+ access channels.
Core abstraction: AutoSci is Wiki-centric — OmegaWiki, a file-based knowledge graph (Markdown entities + YAML frontmatter + JSONL graph edges) is the single source of truth, with 9 entity types and 15 edge types, offering maximal transparency but weak concurrency control and O(N) filesystem queries. EvoScientist is Agent-graph-centric — a deepagents + LangGraph state machine (recursion_limit=1000, six YAML-configured sub-agents, AsyncSqliteSaver checkpoints) with strong execution control but lower knowledge transparency.
Memory: AutoSci requires explicit writes via tools/research_wiki.py, enforces bidirectional links, tool-exclusive graph writes, append-only logs, and lifecycle state machines. EvoScientist uses a three-tier memory (SQLite session checkpoints, MEMORY.md long-term memory, research memory) with automatic LLM-based extraction every 20 messages (UserProfile, ResearchPreferences, ExperimentConclusion, LearnedPreferences).
Runtime: AutoSci's skills are Markdown SKILL.md prompt definitions executed by the closed-source Claude Code runtime; sub-agents use Claude Code's native Agent tool and an MCP-based review LLM. EvoScientist builds an explicit programmable graph with 12 middlewares (memory, HITL, fallback, tool selection, code interpreter) and a unique think_tool forcing 7-dimension metacognitive reflection.
LLM routing: AutoSci uses a simple dual-model setup (main Claude model + OpenAI-compatible review LLM) without runtime switching or fallback. EvoScientist routes across 15+ providers (5 native, OpenAI/Anthropic-compliant routing), with auto-configuration (extended thinking, reasoning effort), compatibility patches, ModelFallbackMiddleware, and runtime /model switching.
Interaction: AutoSci is CLI-only, with read-only visualization (web server on port 8765, Obsidian Canvas). EvoScientist implements an omnichannel Channel abstraction (Telegram, Slack, Discord, WeChat, DingTalk, Feishu, Email, QQ, Signal, iMessage), an asyncio dual-queue message bus, a worker-pool consumer with per-chat locks and LRU session mapping, 13 standardized streaming event types, and a Textual-based TUI.
Security: AutoSci relies on Claude Code's built-in permissions and textual constraints, with no path traversal detection or command blacklist. EvoScientist provides a CustomSandboxBackend with virtual path mapping, system-path interception, traversal detection, dangerous-command blacklists (pipe-aware, shell-split), 300s timeouts, 100KB output caps, sender/channel allowlists, and HITL approvals (1=approve, 2=reject, 3=approve all, 120s timeout auto-approve).
Engineering: AutoSci is a lightweight script collection (19 Python tools; research_wiki.py is ~2861 lines) without packaging, tests, or linting, but with bilingual (en/zh) skills. EvoScientist is a modern Python package with ~890 tests (no API key required), Ruff + pre-commit, CI/CD, non-root Docker images, and auto-generated architecture docs.
Deployment: AutoSci targets a local dev machine with optional GitHub Actions scheduling for /daily-arxiv and remote GPU experiments via ssh/rsync/screen. EvoScientist offers CLI/TUI, headless daemon mode (EvoSci serve with /healthz), a LangGraph server mode (EvoSci deploy), and Docker Compose.
Extensibility: AutoSci extends via YAML schema edits and new SKILL.md files. EvoScientist extends via module registration (channels), model registry entries, YAML sub-agent files, MCP server management (EvoSci mcp add), and a three-layer skill system backed by the companion EvoSkills repo.
Experiment workflows: AutoSci uses explicit command-driven stages (/exp-design, /exp-run, /exp-eval, pilot variants). EvoScientist delegates autonomously among planner/code/debug/data-analysis/writing agents, with stopping criteria including stable metrics across >=3 seeds.

Conclusion

AutoSci optimizes for transparency, human auditability, and Git-friendly knowledge management within an existing Claude Code workflow, at the cost of scalability, robustness, and autonomy. EvoScientist optimizes for autonomous multi-agent execution, operational robustness, and omnichannel access, at the cost of knowledge-graph transparency for human readers.