AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play
Paper: https://www.arxiv.org/abs/2509.24193 Authors: Ran Xu, Yuchen Zhuang, Zihan Dong, Jonathan Wang, Yue Yu, Joyce C. Ho, et al. (10 authors) Published: 2025-09-29 Category: Agentic Search
Overview
Search-augmented LLMs often struggle with complex reasoning tasks due to ineffective multi-hop retrieval and limited reasoning ability. AceSearcher addresses this with a cooperative self-play framework in which a single LLM alternates between two roles:
- Decomposer: breaks down complex queries into manageable sub-questions
- Solver: integrates retrieved contexts for answer generation
- Evaluated on three reasoning-intensive tasks across 10 datasets.
- Outperforms state-of-the-art baselines with an average exact match improvement of 7.6%.
- On document-level finance reasoning, AceSearcher-32B matches DeepSeek-V3 using less than 5% of its parameters.
- At smaller scales (1.5B and 8B), AceSearcher often surpasses existing search-augmented LLMs with up to 9x more parameters.
- Code: https://github.com/ritaranx/AceSearcher
- Models: https://huggingface.co/AceSearcher
- A Systematic Framework for Enterprise Knowledge Retrieval
- Retrieval Augmented Generation and Understanding in Vision: A Survey
- Synergizing RAG and Reasoning: A Systematic Review
- Agentic Information Retrieval
Method
AceSearcher combines two training stages:
1. Supervised fine-tuning (SFT) on a diverse mixture of search, reasoning, and decomposition tasks. 2. Reinforcement fine-tuning (RFT) optimized for final answer accuracy, eliminating the need for intermediate annotations (e.g., per-hop retrieval labels).
This self-play setup bootstraps both query decomposition and context integration abilities within one model, enabling iterative multi-hop search and reasoning without hand-labeled trajectories.
Results
Resources
Why It Matters
AceSearcher demonstrates that reinforced self-play can turn a single mid-sized LLM into an effective agentic search system, achieving strong multi-hop reasoning with far fewer parameters than frontier models—a promising direction for efficient retrieval-augmented generation (RAG) and agentic search deployments.