Dr. Zero: Self-Evolving Search Agents without Training Data
Overview
| Field | Content | |------|------| | Title | Dr. Zero: Self-Evolving Search Agents without Training Data | | Authors | Zhenrui Yue, Kartikeya Upasani, Xianjun Yang, Suyu Ge, Shaoliang Nie, Yuning Mao, et al. (8 authors) | | Published | 2026-01-11 | | Link | https://arxiv.org/abs/2601.07055 | | Type | Academic paper (Agentic Search) |
Key points
- Problem: As high-quality training data becomes increasingly difficult to obtain, data-free self-evolution is emerging as a promising paradigm — but multi-turn search agents struggle with it due to limited question diversity and the substantial compute required for multi-step reasoning and tool use.
- Approach: Dr. Zero enables search agents to self-evolve without any training data via a self-evolution feedback loop:
- A proposer generates diverse questions to train a solver initialized from the same base model.
- As the solver evolves, it incentivizes the proposer to produce increasingly difficult yet solvable tasks, forming an automated curriculum that refines both agents.
- Efficiency: The paper introduces hop-grouped relative policy optimization (HRPO), which clusters structurally similar questions to construct group-level baselines. This minimizes sampling overhead in evaluating each query's difficulty and solvability, significantly reducing compute for solver training without compromising performance or stability.
- Results: Extensive experiments show the data-free Dr. Zero matches or surpasses fully supervised search agents, proving complex reasoning and search capabilities can emerge solely through self-evolution.
- Architecture: Agentic paradigms are turning retrieval frequency and strategy themselves into learnable objectives, beyond cascaded retrieve-rank-generate pipelines.
- Data: Synthetic, self-generated tasks can replace supervised data — but care is needed to avoid distribution drift.
- Engineering: HRPO shows that structural grouping of similar queries is an effective lever for cutting RL training compute.
- Open questions: evaluation trustworthiness, latency/cost, hallucination and safety, cross-lingual and multimodal extension.
- Original paper: Dr. Zero: Self-Evolving Search Agents without Training Data
Original abstract
> As high-quality data becomes increasingly difficult to obtain, data-free self-evolution has emerged as a promising paradigm. This approach allows large language models (LLMs) to autonomously generate and solve complex problems, thereby improving their reasoning capabilities. However, multi-turn search agents struggle in data-free self-evolution due to the limited question diversity and the substantial compute required for multi-step reasoning and tool using. In this work, we introduce Dr. Zero, a framework enabling search agents to effectively self-evolve without any training data. In particular, we design a self-evolution feedback loop where a proposer generates diverse questions to train a solver initialized from the same base model. As the solver evolves, it incentivizes the proposer to produce increasingly difficult yet solvable tasks, thus establishing an automated curriculum to refine both agents. To enhance training efficiency, we also introduce hop-grouped relative policy optimization (HRPO). This method clusters structurally similar questions to construct group-level baselines, effectively minimizing the sampling overhead in evaluating each query's individual difficulty and solvability. Consequently, HRPO significantly reduces the compute requirements for solver training without compromising performance or stability. Extensive experiment results demonstrate that the data-free Dr. Zero matches or surpasses fully supervised search agents, proving that complex reasoning and search capabilities can emerge solely through self-evolution.
Context
The work sits at the intersection of agentic search and large-scale search/recommendation systems. Classic search pipelines separate retrieval, ranking, and generation; in the LLM era, reasoning budget and action space (whether to retrieve, how many steps, which tools) become learnable variables. Dr. Zero addresses the data bottleneck by letting agents bootstrap their own curriculum.