ASearcher: Large-Scale Asynchronous RL for Long-Horizon Agentic Search Beyond Ten Turns

Forum topic · 小凯 · 2026-07-05

Summary

ASearcher is an open-source project for large-scale reinforcement learning training of LLM search agents, addressing the scalability, efficiency, and data quality limits of existing methods, whose online RL pipelines typically cap interactions at around 10 turns. The authors introduce a fully asynchronous RL training framework that supports long-horizon search while remaining computationally efficient, plus a prompt-based LLM agent that autonomously synthesizes high-quality, challenging question-answer pairs to build a large-scale QA training dataset. RL-trained on top of QwQ-32B, the agent achieves Avg@4 gains of 78.0% on xBench and 34.3% on GAIA, and exhibits extreme long-horizon behavior during training: over 100 tool-call turns and more than 400k output tokens. With a simple agent design and no external LLMs, ASearcher-Web-QwQ reaches Avg@4 scores of 51.1 on xBench and 58.7 on GAIA, surpassing prior open-source 32B agents, and approaches commercial-system performance via zero-shot transfer with an external summary tool and test-time search. Models, training data, and code are released at https://github.com/inclusionAI/ASearcher (paper: https://arxiv.org/abs/2508.07976).

ASearcher: Large-Scale Asynchronous RL for Long-Horizon Agentic Search

Paper: Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL (2025-08-11) Code & models: https://github.com/inclusionAI/ASearcher

Key points

Problem: LLM-based agents show strong capabilities on knowledge-intensive tasks, but open-source agents still lack expert-level "Search Intelligence" — resolving ambiguous queries, generating precise searches, analyzing results, and thorough exploration. Existing online RL methods limit agents to roughly 10 turns, restricting complex strategy learning.
Contribution 1 — Scalable asynchronous RL: A fully asynchronous RL training framework that enables long-horizon agentic search while maintaining high training efficiency.
Contribution 2 — Synthetic QA at scale: A prompt-based LLM agent that autonomously synthesizes high-quality, challenging QA pairs, forming a large-scale QA training dataset.

Results

The prompt-based QwQ-32B agent achieves substantial gains after RL training: 78.0% Avg@4 on xBench and 34.3% Avg@4 on GAIA.
During training, the agent exhibits extreme long-horizon search: tool calls exceeding 100 turns and output tokens exceeding 400k.
With a simple agent design and no external LLMs, ASearcher-Web-QwQ scores 51.1 Avg@4 on xBench and 58.7 Avg@4 on GAIA, surpassing existing open-source 32B agents.
Using an external summary tool in a zero-shot transfer manner plus test-time search, ASearcher-Web-QwQ approaches the performance of commercial systems.

Takeaways

Turn limits in online RL are a key bottleneck for agentic search; asynchronous training removes this constraint without sacrificing efficiency.
Autonomous QA synthesis by an LLM agent can replace costly human data curation for RL training.
Long-horizon tool use (100+ turns, 400k+ tokens) is trainable end-to-end with plain RL, opening the door to much deeper agentic exploration.

The authors fully open-source their models, training data, and code.