Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL (ASearcher)

Forum topic · 小凯 · 2026-07-05

Summary

ASearcher is an open-source project for large-scale reinforcement learning training of LLM search agents, introduced to overcome turn limits of existing online RL methods (typically 10 turns or fewer) that restrict learning of complex search strategies. Its key contributions are: (1) a scalable, fully asynchronous RL training framework that enables long-horizon search while maintaining high training efficiency, and (2) a prompt-based LLM agent that autonomously synthesizes high-quality, challenging question-answer pairs to build a large-scale QA training dataset. Training a prompt-based QwQ-32B agent with RL yields substantial gains of 78.0% and 34.3% Avg@4 on xBench and GAIA respectively. The trained agent exhibits extreme long-horizon behavior, with tool calls exceeding 100 turns and outputs exceeding 400k tokens during training. With a simple agent design and no external LLMs, ASearcher-Web-QwQ reaches Avg@4 scores of 51.1 on xBench and 58.7 on GAIA, surpassing existing open-source 32B agents, and can match commercial systems via zero-shot transfer with an external summary tool plus test-time search. Models, training data, and code are open-sourced at https://github.com/inclusionAI/ASearcher; the paper is available at https://arxiv.org/abs/2508.07976.

Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL

Paper: https://arxiv.org/abs/2508.07976 Code / models / data: https://github.com/inclusionAI/ASearcher Authors: Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, et al. (8 authors total) Published: 2025-08-11

Background

LLM-based agents increasingly rely on search tools to access external knowledge for complex, knowledge-intensive tasks. However, open-source agents still fall short of expert-level *Search Intelligence* — the ability to resolve ambiguous queries, generate precise searches, analyze results, and conduct thorough exploration. Existing approaches are limited in scalability, efficiency, and data quality. In particular, small turn limits in existing online RL methods (e.g., ≤10 turns) restrict the learning of complex, long-horizon search strategies.

Key Contributions

ASearcher: an open-source project for large-scale RL training of search agents.
Scalable fully asynchronous RL training that enables long-horizon search while maintaining high training efficiency.
A prompt-based LLM agent that autonomously synthesizes high-quality and challenging QAs, creating a large-scale QA dataset.
Demonstrated extreme long-horizon search: during training, the agent's tool calls exceed 100 turns and output tokens exceed 400k.

Results

The prompt-based QwQ-32B agent achieves substantial improvements after RL training: 78.0% and 34.3% Avg@4 gains on xBench and GAIA, respectively.
With a simple agent design and no external LLMs, ASearcher-Web-QwQ achieves Avg@4 scores of 51.1 on xBench and 58.7 on GAIA, surpassing existing open-source 32B agents.
ASearcher-Web-QwQ can reach the performance of commercial systems via zero-shot transfer using an external summary tool and test-time search.

Takeaways

1. Fully asynchronous RL removes turn-limit bottlenecks, making long-horizon agentic search trainable at scale. 2. Agent-synthesized QA data provides a scalable path to high-quality RL training data without heavy human annotation. 3. Simple agent designs, trained with large-scale RL, can rival or exceed commercial search-agent systems on public benchmarks.

Related Work in This Collection

A Systematic Framework for Enterprise Knowledge Retrieval
Retrieval Augmented Generation and Understanding in Vision: A Survey
Synergizing RAG and Reasoning: A Systematic Review
AceSearcher: Bootstrapping Reasoning and Search for LLMs via RL
Agentic Information Retrieval

*All quantitative figures above are taken from the paper's abstract; consult the PDF for full experimental details.*