ZeroSearch: Training LLM Search Skills via Simulated Search Instead of Real APIs

Forum topic · 小凯 · 2026-07-05

Summary

ZeroSearch (arXiv:2505.04588, May 2025) is a reinforcement learning framework that trains large language models to use search engines without actually querying live search APIs. Prior RL approaches for improving LLM search ability suffer from two problems: unpredictable document quality from real search engines adds noise to training, and frequent rollouts can require hundreds of thousands of API requests, driving up costs. ZeroSearch instead fine-tunes an LLM via lightweight supervised training to act as a simulated retrieval module that generates both useful and noisy documents for a query. During RL training, a curriculum-based rollout strategy progressively degrades document quality, exposing the model to increasingly difficult retrieval scenarios and eliciting stronger reasoning. Experiments show a 3B LLM suffices as the retrieval module; a 7B simulated search module matches a real search engine, and a 14B module surpasses it. The framework generalizes across base and instruction-tuned models of various sizes and is compatible with multiple RL algorithms, offering a cost-controlled, quality-controllable alternative to live-engine RL training for search-capable LLMs.

ZeroSearch: Incentivize the Search Capability of LLMs without Searching

Paper: https://arxiv.org/abs/2505.04588 Authors: Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, et al. (10 authors) Published: 2025-05-07

TL;DR

ZeroSearch is an RL framework that trains LLMs to use search engines by replacing live search APIs with a simulated search engine built from another LLM, avoiding noisy documents and prohibitive API costs during training.

Background

Effective information searching is essential for enhancing the reasoning and generation capabilities of LLMs. Recent research has used reinforcement learning (RL) to improve LLMs' search capabilities by interacting with live search engines, but this faces two major challenges:

1. Uncontrolled Document Quality — documents returned by search engines are unpredictable, introducing noise and instability into training. 2. Prohibitively High API Costs — RL rollouts may involve hundreds of thousands of search requests, severely constraining scalability.

How ZeroSearch Works

1. Simulated retrieval module: Lightweight supervised fine-tuning transforms an LLM into a retrieval module that generates both useful and noisy documents in response to a query. 2. Curriculum-based rollout: During RL training, the quality of generated documents is incrementally degraded, progressively exposing the model to harder retrieval scenarios and eliciting its reasoning ability.

Results

A 3B LLM is sufficient as the retrieval module.
A 7B retrieval module achieves performance comparable to a real search engine.
A 14B retrieval module even surpasses the real search engine.
ZeroSearch generalizes across base and instruction-tuned models of various parameter sizes and is compatible with a wide range of RL algorithms.

Why It Matters

Cost control: eliminates the massive API bills of live-search RL training.
Quality control: document noise becomes a trainable, curriculum-controlled variable rather than an uncontrollable external factor.
Scalability: rollouts run locally on GPUs, removing rate limits and latency bottlenecks.

Original Abstract (excerpt)

> To address these challenges, we introduce ZeroSearch, a novel RL framework that incentivizes the capabilities of LLMs to use a real search engine with simulated searches during training... Extensive experiments demonstrate that ZeroSearch effectively incentivizes the search capabilities of LLMs using a 3B LLM as the retrieval module.

Related Work

ReSearch: Learning to Reason with Search for LLMs via RL (arXiv:2503.19470)
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation (arXiv:2310.03214)
Enhancing Noise Robustness of Retrieval-Augmented Language Models (arXiv:2405.20978)
Interleaving Retrieval with Chain-of-Thought Reasoning (arXiv:2212.10509)
Long-form factuality in large language models (arXiv:2403.18802)

*Quantitative details should be verified against the original PDF before citing specific numbers.*