DeepSeek Researcher's Agent Writes a 46-Page Survey About Itself in 6 Days

Forum topic · 小凯 · 2026-05-29

Summary

DeepSeek core researcher Deli Chen used his own agent framework, DeliAutoResearch, to write a 46-page survey titled 'From Copilots to Colleagues: A Survey of Autonomous Research Agents' — 99% authored by the agent itself. The draft took 76 minutes; full production required 6 days, 108 interaction rounds, and 648,000 tokens, with under 2 hours of human supervision. The survey classifies autonomous research agents into five autonomy levels (L1 autocomplete to L5 self-directed), places today's frontier at L4 (fully autonomous in bounded domains like Devin, SWE-Agent, AI Scientist, Claude Code), and identifies evaluation capability — not model capability — as the main barrier to L5. It analyzes four architecture patterns (single-agent loops, multi-agent systems, hierarchical orchestration, tool-augmented execution), compares 17 representative systems including SWE-bench results, and highlights six open problems: cognitive loops, context limits, novelty evaluation, reproducibility, safety, and cost. The paper itself serves as evidence of the L4 phenomenon it studies.

DeepSeek core researcher Deli Chen (a key contributor to DeepSeek-V1/V2/V3/V4/R1/Coder/MoE architectures) published a 46-page survey, From Copilots to Colleagues: A Survey of Autonomous Research Agents (https://victorchen96.github.io/auto_research_survey.pdf), in which 99% of the text was written by his own agent framework, DeliAutoResearch: first draft in 76 minutes, 6 days and 6 iterations, 108 interaction rounds, 648K tokens, 46 pages, 103 references, 7 figures, 4 tables — with less than 2 hours of human time. The paper is itself an instance of the phenomenon it studies.

L1–L5: Five Steps of Agent Autonomy

The survey classifies autonomous research agents like SAE driving levels:

| Level | Name | Human Role | Autonomous Duration | Examples | |---|---|---|---|---| | L1 | Autocomplete | Full control | None | GitHub Copilot | | L2 | Task execution (approval) | Approve each step | Minutes | ChatGPT + Code Interpreter | | L3 | Multi-step (checkpoints) | Checkpoint review | 10 min–1 hr | Claude Code, AutoGPT | | L4 | Fully autonomous (bounded domain) | Evaluate output only | Hours–days | Devin, SWE-Agent, AI Scientist, Claude Code auto mode | | L5 | Self-directed | Set domain only | Weeks–months | None yet (Co-Scientist approximates) |

Key judgments: the current frontier is L4, but only in bounded domains with clear verifiers (passing tests, successful experiments, proven theorems). The barrier to L5 is not model capability but evaluation capability — no system can reliably judge novelty and importance.

Four Architecture Patterns

Single-agent loops: ReAct interleaves reasoning and action; Reflexion lifts HumanEval from 67% to 91%; LATS with MCTS reaches 94.4% at 5–20x cost. Bottleneck is context saturation — degradation after 10–15 steps.
Multi-agent systems: MetaGPT encodes a software company's SOPs and achieves 100% task completion vs. 67% for free-form chat (ChatDev). Structured, document-mediated communication is key.
Hierarchical orchestration: supervisor–worker patterns isolate context (Claude Code); graph-based orchestration optimizes collaboration topology itself.
Tool-augmented execution: SWE-Agent's Agent-Computer Interface design raises SWE-bench from 3.8% to 12.5%; CodeAct unifies actions as executable Python code.

Findings from 17 Systems

Code agents are the most mature track (clear verifiers, standardized benchmarks): Claude Code 72% on SWE-bench Verified; OpenHands 41–53%; Devin 13.86%.
General agents are least reliable: AutoGPT collapses after 10–15 steps; Agentless — a simple two-step pipeline, not an agent — scores 27% on SWE-bench Lite at far lower cost, raising the question of whether agent architectures can be over-engineering.
Science agents are limited by physical interfaces: Coscientist synthesized aspirin and fluorescent dyes, but long feedback cycles in biology/materials break the iteration loop.
AI Scientist ($15/paper) covers idea generation through automated peer review, but cannot choose what is worth researching — a typical L4 limitation: it can execute, not select.

Six Open Problems

1. Cognitive loops — agents repeat failing patterns instead of reframing problems. 2. Context limitations — effective (not nominal) context length degrades on long-horizon tasks. 3. Novelty evaluation — the core obstacle to L5; no automated solution in open domains. 4. Reproducibility — non-deterministic agents violate scientific norms. 5. Safety — dual-use risks in chemistry/biology; humans cannot supervise hours of autonomous execution in real time. 6. Cost — $5–$50 per L4 run; weeks-long L5 runs would explode compute costs.

The Meta-Narrative

The survey's own production matches its L4 definition: end-to-end autonomous execution with human evaluation of the final output — but not L5, because the research topic was human-given. Chen notes that Code Agents are causing "crazy inflation" of CS papers. When a survey's cost drops from a person-month to 2 hours of human oversight plus 6 days of machine time, output volume swells and quality distribution widens — filtering and verification work increases in reverse. Agents do not eliminate human work; they shift it from writing to selecting and verifying.

> "With CodeAgent, I can finally pick back up many things I shelved due to lack of time." — Deli Chen

Source: From Copilots to Colleagues: A Survey of Autonomous Research Agents, Deli Chen, https://victorchen96.github.io/auto_research_survey.pdf