Deli Chen Open-Sources AutoResearch: AI Agent Runs Full 285B RL Research Loop Autonomously

Forum topic · QianXun · 2026-06-22

Summary

DeepSeek senior researcher Deli Chen (陈德里) has open-sourced Deli AutoResearch SKILL.md, a protocol framework rather than executable code, and released a 75-page fourth survey, Self-Play in the Age of Foundation Models, with 217 citations. The project documents the first time an AI agent ran a complete research loop on the DeepSeek 285B model with zero human intervention: experimental design, coding, GPU job submission, debugging, and conclusion. The agent iterated through 16 review rounds and 12 RL runs, including a 285B GRPO self-play experiment. A core empirical finding is a strong negative correlation between validation-signal noise epsilon and self-play training improvement: +4.8% improvement at epsilon=0 versus -6.6% at epsilon=0.45, with KL anchoring shifting cost placement but not eliminating noise. The protocol formalizes three observed failure modes—cognitive loops, stalls, and runtime fragility—and prescribes a three-layer guardian architecture, hard behavioral constraints, file-based state persistence, and forced directional diversity.

Key points

Autonomous 285B RL research loop: An AI agent ran the full pipeline (design → code → GPU job → debug → conclusion) on DeepSeek 285B with zero human operational input. Six directional human inputs were given across a 72-hour stretch, but no operational intervention.
Core empirical finding from 285B GRPO self-play: Strong negative correlation between validation-signal noise ε and self-play training improvement. ε=0 yielded +4.8% improvement; ε=0.45 yielded −6.6%. KL anchoring did not remove noise, only shifted where the cost falls between training distribution and held-out set.
Review iteration: 16 review rounds, 12 RL runs. Version V11 scored 8.5 after writing up 285B GRPO results; V12 dropped to 8.2 after external literature checks flagged 3 problematic citations; V16 rose to 8.6 after theoretical hardening.
Four-paper output: Total 265 pages and 1,158 citations across *Autonomous Research Agents* (59 pp, 228 cites, 8.0/10), *Continual Learning* (65 pp, 326 cites, 8.0/10), *Long-Horizon Decision-Making* (55 pp, 384 cites, 8.0/10), and *Self-Play in the Age of Foundation Models* (75 pp, 217 cites, 8.6/10).
Three documented failure modes from real long-horizon agent runs:
Architecture: Orchestrator + Worker agents + Guardian layer. Workers run in fresh sessions; state is injected via files, not session history; heartbeat watchdogs in three layers (L0 resident shell, L1 durable cron, L2 in-loop).
Five hard behavioral constraints: zero interaction during runs (no Plan Mode, no asking questions); ready means execute; callback means report-alive; persist state to files; guardian/worker separation (guardian may only liveness-check, restart, or nudge).
Stall detection and pivoting: a single iteration with 0 new findings or metric regression increments stale_count; stale_count ≥ 2 forces a structural-constraint pivot; ≥ 4 flags human intervention; new directions must differ from all historical attempts; per-session limit is ≤15 rounds or 30 minutes.
Six engineering constraints: ≤5 large files per iteration, each ≤300 lines; state via files; mandatory validation between iterations; citation verification every 20 references; prioritize diversity over depth on multiple candidates; unrecoverable external failures must escalate (full report + owner notification + polling reply), never silently abort.
Paper-writing sub-flows: literature, structure, experiment, charts, mock review. Quality gates require four check classes to pass plus clean PDF compile, review-score threshold, and non-regression of fixed issues before stage advancement.

Why it matters

Marks a transition from "AI assists paper writing" to "AI autonomously runs full research loops" — described as a milestone for Agent-for-Science.
The framework's value is engineering, not theoretical elegance: every mechanism targets a real observed failure mode.
SKILL.md is a protocol, not a code dependency, making it portable across teams and infrastructure.
The 285B finding that validation-signal reliability is the bottleneck for self-play directly challenges scaling assumptions behind current RLHF pipelines.
Open release lowers reproduction barriers.

Risks and open questions

Self-assigned scores are internally consistent (vertical comparability) and come from in-framework mock review, not external peer review; 8.6/10 is not absolute quality.
LLM-generated citations and data artifacts remain a source of error; the framework mechanizes checks every 20 references but does not eliminate the underlying risk.
Separation of responsibilities relies on protocol constraints rather than model self-discipline.
285B experiment details continue to be audited — V12 still surfaced 3 problematic citations, so residual error is realistic.
Reproduction cost is high: 12 RL runs, 16 review rounds, 285B-scale training, and a 72-hour sustained run exceed what most teams can afford.

Project page: https://victorchen96.github.io/auto_research/framework.html