ARIS: An Open-Source Framework for Autonomous AI Research via Adversarial Multi-Agent Collaboration

Forum topic · 小凯 · 2026-06-07

Summary

Researchers at Shanghai Jiao Tong University propose ARIS, an open-source autonomous research framework designed to tackle the core failure mode of long-horizon AI agents: 'plausible unsupported success,' where agents produce fluent but fabricated or overgeneralized research claims. ARIS decouples execution, orchestration, and assurance into three layers, using cross-model adversarial collaboration between an Executor and an independent Reviewer (from a different model family) in a critique-to-action loop. The system ships 65+ portable Markdown skills compatible with Claude Code, Codex CLI, Cursor, and others; a persistent research wiki knowledge graph that prevents repeated failed explorations; a three-stage claim-integrity audit (experiment integrity, result-to-claim mapping, paper-claim audit); and a five-round scientific editing pipeline. A documented overnight run lasted about 8 hours with 4 review-revision rounds, improving internal reviewer scores from 5.0 to 7.5/10 while pruning unsupported claims. ARIS contrasts with single-agent pipelines like AI Scientist by engineering trustworthiness as a first-class system component rather than an afterthought, supporting 6+ executor platforms, 6+ reviewer models, and multiple GPU backends. Project: https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep

A team at Shanghai Jiao Tong University (Ruofeng Yang, Yongcan Li, Shuai Li) introduces ARIS (Autonomous Research via Adversarial Multi-Agent Collaboration), an open-source framework for end-to-end AI-driven research — from idea discovery through experiments, paper writing, and rebuttal.

Key Points

Core thesis: In long-horizon agent workflows, the dominant failure mode is not explicit crash but "plausible unsupported success" — agents produce fluent papers whose claims are fabricated, cherry-picked, or overgeneralized. The paper's strict assumption: *any long-horizon task executed by a single agent is unreliable*.
Adversarial collaboration: A cross-model Executor/Reviewer loop (the reviewer runs on a different model family) is presented as the minimal configuration to break self-play blind spots. The reviewer reads artifact files directly (no executor summaries), and fresh-context review threads prevent confirmation bias.
Three-layer architecture:
Execution layer: 65+ pure-Markdown skills (portable across Claude Code, Codex CLI, Cursor, Trae, etc.), MCP/tool bridges (DBLP, CrossRef, Semantic Scholar, Zotero, W&B, Feishu), a research wiki, and a deterministic FigureSpec renderer.
Orchestration layer: five end-to-end workflows (idea discovery, experiment bridge, auto review loop, paper writing, rebuttal) plus effort levels (lite ≈0.4×, balanced 1×, max ≈2.5×, beast ≈5–8×); reviewer reasoning effort stays at maximum regardless of preset.
Assurance layer: three-stage claim-integrity audit (experiment integrity, result-to-claim mapping into a claim ledger, zero-context paper-claim audit) plus a five-round scientific editing pipeline (clutter removal, active voice, sentence structure, terminology consistency, numerical consistency).
Persistent memory: a research wiki with four entity types (Papers, Ideas, Experiments, Claims) and eight relation types (e.g., extends, contradicts, invalidates), enabling spiral learning instead of repeating failed ideas across sessions.
Documented overnight run: ~8 hours, 4 full review-revision cycles, internal reviewer score improving 5.0 → 7.5/10, 20+ GPU experiments launched, and unsupported claims actively pruned during audit.
Contrast with prior work: unlike single-agent pipelines (e.g., AI Scientist), ARIS targets credibility via multi-agent mutual critique rather than speed.

Noted Limitations

High compute cost versus single-agent approaches
Dependence on strong (often closed-source) reviewer models
No arbitration layer above the "rescue agent" if the third independent diagnostic model also fails
Three-stage audits reduce but cannot eliminate reviewer hallucination
Current validation is concentrated in ML/CS domains

Reference

Yang, R., Li, Y., & Li, S. (2026). *ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration*. Shanghai Jiao Tong University, Shanghai Innovation Institute. arXiv:2605.03042.

Project: https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep