Dr. Zero: Self-Evolving Search Agents without Training Data
Paper: Dr. Zero: Self-Evolving Search Agents without Training Data (arXiv, 2026-01-11) Authors: Zhenrui Yue, Kartikeya Upasani, Xianjun Yang, Suyu Ge, Shaoliang Nie, Yuning Mao, et al. (8 authors total) Category: Agentic Search
Overview
As high-quality training data becomes increasingly difficult to obtain, data-free self-evolution has emerged as a promising paradigm. Dr. Zero enables multi-turn search agents built on large language models (LLMs) to improve their reasoning and search capabilities without any training data.
Existing data-free self-evolution approaches struggle with multi-turn search agents for two reasons:
- Limited diversity of generated questions
- The substantial compute required for multi-step reasoning and tool use
- Clusters structurally similar questions to construct group-level baselines
- Minimizes the sampling overhead of evaluating each query's individual difficulty and solvability
- Significantly reduces the compute required for solver training without compromising performance or stability
- Architecture: agentic paradigms treat retrieval frequency and strategy as learnable objects rather than fixed pipelines.
- Data: synthetic, self-generated data can substitute for supervised training data when the curriculum keeps tasks difficult yet solvable.
- Efficiency: group-level baselines over clustered questions (HRPO) are a practical recipe for cutting RL compute in agentic settings.
- Quantitative results should be verified against the original PDF tables; this summary is based on the abstract and public metadata.
- Generalization across languages and real user distributions, plus safety risks of open-web retrieval, remain open questions.
- Dr. Zero: Self-Evolving Search Agents without Training Data. https://arxiv.org/abs/2601.07055
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
- Agentic Information Retrieval (arXiv, Oct 2024)
- Synergizing RAG and Reasoning: A Systematic Review (arXiv, Apr 2025)
Method
Self-Evolution Feedback Loop
Dr. Zero uses two agents initialized from the same base model:
1. Proposer — generates diverse questions to train the solver. 2. Solver — learns to answer via multi-turn search; as it evolves, it incentivizes the proposer to produce increasingly difficult yet solvable tasks.
This establishes an automated curriculum that continuously refines both agents.
Hop-Grouped Relative Policy Optimization (HRPO)
To improve training efficiency, the authors propose HRPO, which:
Results
Extensive experiments show that the fully data-free Dr. Zero matches or surpasses fully supervised search agents, proving that complex reasoning and search capabilities can emerge solely through self-evolution.
> As high-quality data becomes increasingly difficult to obtain, data-free self-evolution has emerged as a promising paradigm. This approach allows large language models (LLMs) to autonomously generate and solve complex problems, thereby improving their reasoning capabilities. However, multi-turn search agents struggle in data-free self-evolution due to the limited question diversity and the substantial compute required for multi-step reasoning and tool using. In this work, we introduce Dr. Zero, a framework enabling search agents to effectively self-evolve without any training data. [...] Extensive experiment results demonstrate that the data-free Dr. Zero matches or surpasses fully supervised search agents, proving that complex reasoning and search capabilities can emerge solely through self-evolution.
Discussion and Context
The paper sits at the intersection of agentic search and large-scale retrieval systems. Within the classic search stack (recall → ranking → generation), the LLM era introduces new variables: reasoning budget and action space — whether to retrieve, how many times, and which tools to call. Dr. Zero's contribution is showing that the retrieval policy itself can be learned through self-play without labeled data.