Agent0: Self-Evolving AI Agents via Tool-Integrated Reasoning with Zero External Data

Forum topic · ✨步子哥 · 2025-11-26

Summary

Agent0 and Agent0-VL are self-evolving agent frameworks introduced in two papers (arXiv:2511.16043 and arXiv:2511.19900) that eliminate dependence on human-annotated training data. Agent0 uses a dual-agent co-evolution mechanism: a curriculum agent generates challenging tasks (rewarded for maximizing executor uncertainty and tool usage), while an executor agent solves them via tool-integrated multi-turn reasoning with majority-vote pseudo-labels. Agent0-VL extends this to vision-language reasoning with a unified solver-verifier architecture and a Self-Evolving Reasoning Cycle (SERC), where tool-based step verification triggers self-repair when confidence falls below a threshold. Reported results on Qwen3-8B include +22.6% on MATH and +11.6% on MMLU-Pro, outperforming R-Zero, Absolute Zero, and Socratic-Zero. Agent0-VL improves Qwen2.5-VL-7B by +11.5% on MathVista and +12.2% on HallusionBench, reducing visual hallucinations. Ablations confirm tool use and the SERC loop are critical. Limitations include computational overhead, tool reliability dependency, and cross-domain generalization. Code is available at https://github.com/aiming-lab/Agent0.

Agent0: Self-Evolving Agents via Tool-Integrated Reasoning with Zero Data

This post summarizes two related papers on self-evolving AI agents that require no human-annotated training data:

1. *Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning* (arXiv:2511.16043) 2. *Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning* (arXiv:2511.19900)

Background and Challenges

Data dependency: Traditional agents rely on human-labeled data (e.g., RLHF), which is costly and bounded by human knowledge.
Capability ceiling: Self-evolution is limited by the model's inherent knowledge, making it hard to generate tasks beyond its own abilities.
Single-turn limitations: Most existing frameworks handle only single-turn interactions, failing at multi-step, tool-dependent dynamic tasks.

Key questions: How to achieve continuous agent evolution with zero external data? How to generate a high-difficulty curriculum beyond the model's fixed capabilities? How to integrate tool use with multi-turn reasoning?

Technical Framework

Agent0: Zero-Data Self-Evolution (Dual-Agent Co-Evolution)

| Role | Function | Training objective | |---|---|---| | Curriculum Agent | Generates challenging tasks; reward based on executor uncertainty and tool-use frequency | Maximize task difficulty (GRPO) | | Executor Agent | Solves tasks with tools; pseudo-labels via majority voting | Minimize task failure rate (ADPO) |

Core innovations:

Tool reward driving curriculum generation:

R_{\text{tool}} = \gamma \cdot \min(N_{\text{tool}}, C)

where $N_{\text{tool}}$ is the number of tool calls.

Dynamic curriculum filtering: keeps tasks with self-consistency $\hat{p} \in [0.3, 0.8]$ for moderate difficulty.
Multi-turn interaction: supports context-dependent conversational tasks (e.g., code debugging).

Agent0-VL: Multimodal Extension

A unified dual-role architecture:

Solver: multi-turn tool calls (e.g., Python sandbox), producing visual reasoning trajectories.
Verifier: evaluates step correctness, produces structured feedback (score, confidence, critique), and triggers self-repair.

Self-Evolving Reasoning Cycle (SERC): 1. Inner loop: solver generates trajectory → verifier checks with tools → repair triggered when confidence is below threshold. 2. Outer loop: GRPO updates policy via process rewards, aligning reasoning and evaluation distributions.

Repair gating:

g_t = \sigma(\kappa (\tau_c - \text{conf}_t))

Repair activates when $\text{conf}_t < \tau_c$ , preventing error propagation.

Experimental Results

Agent0 (Qwen3-8B)

| Benchmark | Improvement | Comparison | |---|---|---| | MATH | +22.6% | Exceeds R-Zero by 6.4%, Absolute Zero by 10.6% | | MMLU-Pro | +11.6% | Exceeds Socratic-Zero by 3.7% | | Tool-use efficiency | +40% call rate growth | Task pass rate 64%→51% over 3 iterations (difficulty rising) |

Agent0-VL (Qwen2.5-VL-7B)

| Benchmark | Improvement | Advantage | |---|---|---| | MathVista | +11.5% | Tool verification reduces visual hallucinations | | HallusionBench | +12.2% | Significant spatial reasoning gains | | Iterative evolution | +12.5% over 3 rounds | Verifier cuts error rate by 25% |

Ablations: removing tool use costs 6.5% (Agent0); removing self-repair costs 2.5% (Agent0-VL); removing SERC costs 8.7% (Agent0-VL).

Use Cases

Geometry problem solving (Agent0-VL): initial blind-quadrant misjudgment → verifier tool detection → logic repair → correct navigation distance calculation.
Math curriculum generation (Agent0): iteration 1 basic algebra → iteration 3 constrained optimization.
Application domains: scientific research (visual data analysis, e.g., astronomy images — Agent0-VL), automated programming (code and test generation — Agent0), edtech (adaptive math question banks — Agent0).

Limitations and Future Directions

Compute overhead: multi-turn interaction and tool calls increase latency (+30% per task for Agent0-VL).
Tool dependency: external tool reliability (e.g., sandbox errors) affects stability.
Generalization: cross-domain transfer (e.g., medical imaging → financial charts) needs validation.

Future work: lightweight tool integration, cross-modal curriculum generation (audio/video), and sparse human feedback for alignment.

Conclusion

Agent0 and Agent0-VL break through data dependency and the capability ceiling via tool-integrated reasoning and zero-data self-evolution. Agent0 lifts general task ability substantially (math and general reasoning gains per tables above), while Agent0-VL resolves multimodal reasoning bottlenecks through tool-verified self-repair. The work offers a reproducible path toward autonomous, self-improving AI agents.

Appendix

Code: https://github.com/aiming-lab/Agent0
Setup: 8× NVIDIA H200, GRPO group size $$G=8$$ , confidence threshold $\tau_c=0.7$