MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability
| Field | Detail | |---|---| | Paper | MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability | | Authors | Weiqi Wu, Xin Guan, Shen Huang, Yong Jiang, Pengjun Xie, Fei Huang, et al. (9 authors) | | Published | 2025-05-26 | | Source | https://arxiv.org/abs/2505.20285 | | Type | Academic paper — Agentic Search |
TL;DR
MaskSearch pre-trains LLM agents to search universally by teaching them to fill masked spans using search tools (the RAMP task), then fine-tunes with SFT and RL (DAPO) for strong gains on multi-hop QA, including out-of-domain tasks.
Background and Motivation
Retrieval-Augmented Language Models (RALMs) enhance generation with external knowledge retrieved via a specialized module. Recent agent techniques let LLMs autonomously use tools for retrieval, planning, and reasoning. However, existing training-based methods show limited agentic ability because they rely on task-specific data. Traditional pipelines also split retrieval, ranking, and generation, poorly matching modern demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. MaskSearch targets universal search capability rather than a single task.
The MaskSearch Framework
Pre-training: RAMP (Retrieval Augmented Mask Prediction)
The model learns to leverage search tools to fill masked spans in large amounts of pre-training data, acquiring universal retrieval and reasoning capabilities for LLMs.
Post-pre-training: SFT + RL
- SFT: training data is generated by combining agent-based and distillation-based methods — a multi-agent system (planner, rewriter, observer) followed by a self-evolving teacher model.
- RL: DAPO serves as the training framework, with a hybrid reward combining answer rewards and format rewards.
- Curriculum learning: instances progress from easier to harder based on the number of masked spans.
- A Systematic Framework for Enterprise Knowledge Retrieval
- Retrieval Augmented Generation and Understanding in Vision: A Survey
- Synergizing RAG and Reasoning: A Systematic Review
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via RL
- Agentic Information Retrieval
- MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability. arXiv:2505.20285
After pre-training, the model is further trained on downstream tasks for additional gains.
Evaluation
The framework is evaluated on open-domain multi-hop question answering. Extensive experiments show MaskSearch significantly enhances LLM-based search agents on both in-domain and out-of-domain downstream tasks. Exact numbers should be checked against the original paper's tables.
Key Takeaways for Search / Rec / Personalization
1. Architecture: cascaded retrieve-rerank-generate remains mainstream, but the agentic paradigm makes retrieval count and policy themselves learnable. 2. Data: high-quality instruction data matters as much as logs; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: offline metrics diverge from online satisfaction; LLM-as-judge should be cross-validated with human evaluation. 4. Product: latency, cost, explainability, and safety are hard constraints for industrial deployment — do not optimize academic benchmarks alone.
Limitations and Future Work
Potential limitations include compute-bound experiment scale, benchmark-vs-real-user distribution mismatch, English-centric data limiting cross-lingual generalization, and safety risks of agents operating on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs/structured databases, and causal/fairness constraints for recommender systems.