EvoScientist Architecture Design: A Human-on-the-Loop Multi-Agent System for Automated Scientific Research

Forum topic · ✨步子哥 · 2026-06-01

Summary

EvoScientist (v0.0.3) is an open multi-agent AI system built on deepagents, LangGraph, and LangChain, designed to autonomously run the full scientific research pipeline—from problem definition and literature review to experiment design, coding, and paper writing. Its design philosophy is Human-on-the-Loop: the AI acts as an autonomous research partner while humans supervise key decision points. Core principles include baseline-first iteration, one-variable-at-a-time experiments, never fabricating results, and aggressive delegation to six specialized sub-agents (planner, research, code, debug, data-analysis, writing), each declared in YAML. The architecture spans a channel layer (Telegram, Slack, Discord, WeChat, and more) decoupled via an async message bus, an inbound consumer with per-chat locks and worker pooling, a main agent with structured reflection via a think_tool, a CompositeBackend virtual filesystem with sandbox path sanitization and dangerous-command blocking, a multi-provider LLM layer with auto-configuration for thinking/reasoning modes, and SQLite session checkpoints. Self-evolution is driven by a three-tier memory system that injects and auto-extracts user profiles, research preferences, and experiment conclusions. The system supports HITL shell-command approval, MCP tool extensions with a marketplace, skills-as-workflows, and CLI, daemon, and programmatic deployment modes.

EvoScientist Architecture Design

EvoScientist is a multi-agent AI system for automated scientific experimentation and discovery (v0.0.3), built on deepagents + LangGraph + LangChain, following a Human-on-the-Loop paradigm.

Key points

Human-on-the-Loop: The AI runs as an autonomous research partner—handling problem definition, literature review, experiment design, implementation, and paper writing—while humans supervise key decision nodes.
Core principles: baseline-first iteration, one core variable changed per iteration, never fabricate results (declare failures honestly), and aggressive delegation of subtasks to specialized sub-agents.
Self-evolution via three-tier memory: SQLite checkpoints (session), /memory/MEMORY.md (long-term user profile and preferences), and research memory files (ideation-memory.md, experiment-memory.md) recording validated/failed directions.
Skills as workflows: skills are first-class citizens (e.g., idea-tournament, experiment-pipeline, paper-writing), each defined in a SKILL.md; agents follow skill workflows rather than improvising, and skills can be installed dynamically.

System architecture

The system is organized in layers:

1. Channel layer: 10 messaging channels (Telegram, Slack, Discord, WeChat, DingTalk, Feishu, Email, QQ, Signal, iMessage) with a plugin-style Channel base class, inbound middleware chain (Dedup → AllowList → Pairing → GroupHistory → MentionGating), auto chunking with code-fence awareness, and exponential backoff retries. Each channel subpackage follows a three-file convention: channel.py, probe.py, serve.py. 2. Message bus: lightweight async dual queues (inbound/outbound, maxsize 5000) decoupling producers from consumers. 3. Consumer layer (InboundConsumer): worker pool (default 5) with bounded-queue backpressure, per-chat serialization locks, LRU session mapping (10,000 cap), HITL interrupt handling (approval via channel reply, 120s auto-approve timeout), ask_user handling, and idle-timeout protection. 4. Agent layer: the main agent is built via deepagents.create_deep_agent() with six sub-agents declared in subagent.yaml:

| Sub-agent | Role | |---|---| | planner-agent | experiment planning and stage reflection | | research-agent | literature and method search (tavily_search) | | code-agent | minimal, reproducible code implementation | | debug-agent | reproduce → root cause → minimal fix | | data-analysis-agent | compute → plot → interpret | | writing-agent | report writing without fabricated results |

A think_tool provides structured reflection across seven dimensions: progress, evidence quality, skill utilization, prior knowledge, strategy, handoff, and resources/compute. The agent graph uses recursion_limit: 1000. 5. Backend layer: CompositeBackend routes / to a sandboxed workspace, /skills/ to a merged read-only skills directory, and /memory/ to a writable filesystem backend. The sandbox sanitizes LLM path hallucinations, blocks path traversal and dangerous commands (sudo, chmod, mkfs, dd, etc.), enforces a 300s execution timeout, and truncates output at 100 KB. 6. LLM layer: get_chat_model() routes to native providers (Anthropic, OpenAI, Google GenAI, NVIDIA, Ollama) and OpenAI/Anthropic-compatible providers (DeepSeek, SiliconFlow, OpenRouter, ZhipuAI, Volcengine, DashScope, MiniMax), with auto-configuration for extended thinking/reasoning and compatibility patches for proxies. 7. Session layer: async SQLite checkpoints at ~/.config/evoscientist/sessions.db with thread CRUD and prefix-matching recovery. 8. Stream layer: 13 normalized event types (thinking, text, tool_call/result, subagent lifecycle, done, usage_stats, interrupt, ask_user, summarization, error).

Core subsystems

Memory system (EvoMemoryMiddleware): injects MEMORY.md into the system prompt each call; every 20 human messages (configurable), an LLM with structured output extracts user profile, research preferences, experiment conclusions, and learned preferences, merged incrementally with deduplication.
HITL system: two levels—shell execution approval gates (interrupt_on={"execute": True} with allowlists) and ask_user middleware supporting text and multiple-choice questions via LangGraph interrupt().
MCP system: manages Model Context Protocol servers via ~/.config/evoscientist/mcp.yaml (stdio/http/sse/websocket transports, env-var interpolation, per-server tool whitelists with glob support, expose_to agent targeting) plus a marketplace registry with 10-minute cache TTL.
Configuration: four-level precedence—CLI args > environment variables > YAML config file > defaults, with 30+ env mappings and .env auto-loading.

Security model

Path safety: virtual path mapping, interception of 12 system path prefixes, path-traversal detection, and auto-correction of hallucinated absolute paths.
Command safety: dangerous-command blacklist with pipe-aware splitting across &&, ||, ;, |.
Access control: sender/channel allowlists, three DM policies (open/allowlist/pairing), mention gating, human approval for shell commands, read-only skills directory.

Extensibility

Extension points include: new channels (implement Channel subclass + auto-registration), new LLM providers (add model entry + routing list + env mapping), new sub-agents (YAML declaration), new MCP servers (config or /mcp add CLI), and new skills (directory + SKILL.md).

Deployment topologies

CLI mode (default): TUI (Textual) or CLI (Rich) with in-process agent runtime.
Daemon mode (evosci serve): multi-channel concurrent access with worker pool, per-chat locks, /healthz endpoint, shared webhook server, graceful shutdown, and exponential-backoff reconnection (1s → 60s max).
Programmatic: stream state snapshots via EvoScientist_agent.stream(...) in notebooks or libraries.

Technology stack

deepagents (≥0.4.11) for agent graphs/backends/middleware; LangGraph for state machines, checkpoints, and interrupts; LangChain (≥1.2.12) as the unified ChatModel interface; provider adapters (Anthropic/OpenAI/Google-GenAI/NVIDIA/Ollama); langchain-mcp-adapters for MCP; aiosqlite for persistence; Typer/Rich/Textual for CLI; PyYAML/python-dotenv for config; httpx for HTTP; faster-whisper (optional) for audio transcription; pytest and Ruff for quality.