AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play

AceSearcher addresses a core weakness of search-augmented LLMs: ineffective multi-hop retrieval and limited reasoning ability on complex questions. It uses a cooperative self-play setup in which one LLM alternates between two roles:

• Decomposer — breaks a complex query into manageable sub-queries.
• Solver — integrates retrieved contexts to produce the final answer.

Training approach

1. Supervised fine-tuning on a diverse mixture of search, reasoning, and query decomposition tasks. 2. Reinforcement fine-tuning optimized only for final answer accuracy, eliminating the need for intermediate annotations.

Results

• Evaluated on three reasoning-intensive tasks across 10 datasets.
• Outperforms state-of-the-art baselines with an average 7.6% exact match improvement.
• On document-level finance reasoning, AceSearcher-32B matches DeepSeek-V3 while using less than 5% of its parameters.
• At smaller scales (1.5B and 8B), AceSearcher often surpasses search-augmented LLMs with up to 9x more parameters, highlighting strong parameter efficiency.

Resources

• Paper: <https://www.arxiv.org/abs/2509.24193>
• Code: <https://github.com/ritaranx/AceSearcher>
• Models: <https://huggingface.co/AceSearcher>

Context

AceSearcher sits at the intersection of retrieval-augmented generation (RAG) and agentic search. Rather than treating retrieval as a one-shot step in a fixed pipeline, it makes both query decomposition and retrieval strategy learnable through self-play — a trend in which the number and policy of retrieval actions themselves become trainable. Reported figures come from the paper abstract; consult the PDF for detailed per-dataset results and ablations.