Agent0: Self-Evolving AI Agents via Tool-Integrated Reasoning with Zero Data

Research Report: Self-Evolving Agents — A Zero-Data Framework for Tool-Integrated Reasoning

Core papers 1. *Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning* (arXiv:2511.19900) 2. *Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning* (arXiv:2511.16043)

---

1. Background and Challenges

1.1 Existing bottlenecks

• Data dependency: Traditional agents rely on human-labeled data (e.g., RLHF), which is costly and bounded by human knowledge.
• Capability ceiling: Model self-evolution is limited by inherent knowledge, making it hard to generate tasks beyond the model's own ability.
• Single-turn interaction limits: Most existing frameworks handle only single-turn interactions and cannot manage multi-step, tool-dependent dynamic tasks.

1.2 Key questions

> How can agents continuously evolve with zero external data? > How can models break through their inherent capability limits to generate high-difficulty curricula? > How can tool use and multi-turn reasoning be integrated to solve complex tasks?

---

2. Technical Framework and Innovations

2.1 Agent0: Zero-data self-evolution

Dual-agent co-evolution mechanism:

| Role | Function | Training objective | |------|----------|-------------------| | Curriculum agent | Generates challenging tasks; reward based on executor uncertainty and tool-use frequency | Maximize task difficulty (GRPO optimization) | | Executor agent | Solves tasks using tools; generates pseudo-labels via majority voting | Minimize task-solving error rate (ADPO optimization) |

Core innovations:

• Tool-reward-driven curriculum:
\[R_{\text{tool}} = \gamma \cdot \min(N_{\text{tool}}, C)\]

where \(N_{\text{tool}}\) is the number of tool calls, incentivizing the curriculum agent to generate complex tasks requiring tools.

• Dynamic curriculum filtering: Only tasks with self-consistency \($\hat{p} \in [0.3, 0.8]$\) are retained to keep difficulty appropriate.
• Multi-turn interaction: Supports context-dependent conversational tasks (e.g., code debugging).

2.2 Agent0-VL: Multimodal self-evolution

Unified dual-role architecture:

| Role | Function | |------|----------| | Solver | Multi-turn tool calls (e.g., Python sandbox), producing visual reasoning trajectories | | Verifier | Evaluates step correctness, generates structured feedback (scores, confidence, critique), and triggers self-repair |

Self-Evolving Reasoning Cycle (SERC): 1. Inner loop: Solver generates trajectories → verifier validates with tools → repair triggered when confidence falls below a threshold. 2. Outer loop: GRPO updates policies using process rewards, aligning reasoning and evaluation distributions.

Tool verification mechanism:

• Repair gating:
\[g_t = \sigma(\kappa (\tau_c - \text{conf}_t))\]

Repair is activated when confidence \($\text{conf}_t < \tau_c$\), preventing error propagation.

---

3. Experimental Results

3.1 Agent0 (math and general reasoning, Qwen3-8B)

| Benchmark | Improvement | Key comparison | |-----------|-------------|----------------| | Math reasoning (MATH) | +22.6% | Exceeds R-Zero by 6.4%, Absolute Zero by 10.6% | | General reasoning (MMLU-Pro) | +11.6% | Exceeds Socratic-Zero by 3.7% | | Tool-use efficiency | +40% call-rate growth | Task pass rate falls from 64% → 51% at iteration 3 (increasing difficulty) |

3.2 Agent0-VL (Qwen2.5-VL-7B)

| Benchmark | Improvement | Key advantage | |-----------|-------------|----------------| | MathVista | +11.5% | Tool verification reduces visual hallucination | | HallusionBench | +12.2% | Notable gains in spatial reasoning | | Iterative evolution | +12.5% over 3 rounds | Verifier cuts error rate by 25% via repair |

Ablation findings:

• Removing tool use → −6.5% (Agent0)
• Removing self-repair → −2.5% (Agent0-VL)
• Removing the SERC cycle → −8.7% (Agent0-VL)

---

4. Use Cases

• Geometry problem solving (Agent0-VL): An initial error (misjudging a blind quadrant) is detected by tool-based verification, repaired, and the navigation distance is correctly computed.
• Math curriculum generation (Agent0): Iteration 1 produces basic algebra; iteration 3 generates constrained optimization problems.

Application domains:

| Domain | Framework | Example | |--------|-----------|---------| | Scientific research | Agent0-VL | Visual data analysis (e.g., astronomical image reasoning) | | Automated programming | Agent0 | Code generation and test-case generation | | EdTech | Agent0 | Adaptive math problem generation |

---

5. Limitations and Future Directions

Current limitations:

• Compute overhead: Multi-turn interaction and tool calls increase inference latency (+30% per task for Agent0-VL).
• Tool dependency: External tool reliability affects stability (e.g., sandbox execution errors).
• Generalization bounds: Cross-domain transfer (e.g., medical imaging → financial charts) needs further validation.

Future directions: 1. Lightweight tool integration (dedicated hardware to reduce latency). 2. Cross-modal curriculum generation (extending to audio and video). 3. Human preference alignment via sparse human feedback to improve curriculum quality.

---

6. Conclusion

Agent0 and Agent0-VL break through the data dependency and capability ceilings of conventional agents via tool-integrated reasoning and zero-data self-evolution:

• Agent0: Dual-agent co-evolution delivers major gains in general task capability (math +22.6%, general reasoning +11.6% on Qwen3-8B).
• Agent0-VL: Vision-language tool verification and self-repair solve multimodal reasoning bottlenecks (average +12.5%).

Core value: A reproducible technical path toward autonomous, self-improving agents without human intervention.

---

Appendix:

• Code: https://github.com/aiming-lab/Agent0
• Experimental setup: 8× NVIDIA H200, GRPO group size \(G=8\), confidence threshold \(\tau_c=0.7\).