MindSearch: Mimicking Human Minds Elicits Deep AI Searcher
- Paper: https://arxiv.org/abs/2407.20183
- Authors: Zehui Chen, Kuikun Liu, Qiuchen Wang, Jiangning Liu, Wenwei Zhang, Kai Chen, et al. (7 authors)
- Published: 2024-07-29
- Category: Agentic Search
- WebPlanner: Models multi-step information seeking as a dynamic graph construction process. It decomposes the user query into atomic sub-questions (graph nodes) and progressively extends the graph based on search results from WebSearcher.
- WebSearcher: For each sub-question, performs hierarchical information retrieval with search engines and collects valuable information for the planner.
- Significant improvement in response quality (depth and breadth) on both closed-set and open-set QA problems.
- Human evaluators preferred responses from MindSearch built on InternLM2.5-7B over ChatGPT-Web and Perplexity.ai, indicating a competitive open solution to proprietary AI search engines.
- A Systematic Framework for Enterprise Knowledge Retrieval
- Retrieval Augmented Generation and Understanding in Vision: A Survey
- Synergizing RAG and Reasoning: A Systematic Review
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
- Agentic Information Retrieval
Overview
MindSearch is an LLM-based multi-agent framework that mimics how humans seek and integrate web information. It targets three key challenges when combining LLMs with search engines:
1. Complex requests often cannot be accurately and completely retrieved in a single search-engine call; 2. The information to be integrated is spread over multiple web pages mixed with massive noise; 3. Large numbers of long web pages quickly exceed the LLM's maximum context length.
Architecture
The multi-agent design allows parallel seeking and integration of information from larger-scale sources (e.g., more than 300 web pages) in 3 minutes — equivalent to roughly 3 hours of human effort.
Results
Key Takeaways for Search/Rec/Personalization
1. Architecture: Cascade retrieval + rerank + generation remains mainstream, but the agentic paradigm makes retrieval count and strategy themselves learnable; 2. Data: High-quality instruction data and click/session logs are both critical; synthetic data requires care against knowledge leakage and distribution shift; 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation; 4. Product: Latency, cost, interpretability, and safety are hard constraints for industrial deployment.
Limitations and Future Directions
Possible limitations include experiment scale constrained by GPU budget, mismatch between benchmarks and real user distributions, unknown cross-lingual generalization from English-centric data, and safety risks of agent systems on the open web. Future work may explore more efficient test-time compute allocation, deeper integration with knowledge graphs/structured databases, and causal/fairness constraints for recommendation.
Related Entries
Original Abstract
> Information seeking and integration is a complex cognitive task that consumes enormous time and effort. Inspired by the remarkable progress of Large Language Models, recent works attempt to solve this task by combining LLMs and search engines. However, these methods still obtain unsatisfying performance due to three challenges: (1) complex requests often cannot be accurately and completely retrieved by the search engine once (2) corresponding information to be integrated is spread over multiple web pages along with massive noise, and (3) a large number of web pages with long contents may quickly exceed the maximum context length of LLMs. Inspired by the cognitive process when humans solve these problems, we introduce MindSearch to mimic the human minds in web information seeking and integration, which can be instantiated by a simple yet effective LLM-based multi-agent framework. The WebPlanner models the human mind of multi-step information seeking as a dynamic graph construction process: it decomposes the user query into atomic sub-questions as nodes in the graph and progressively extends the graph based on the search result from WebSearcher. Tasked with each sub-question, WebSearcher performs hierarchical information retrieval with search engines and collects valuable information for WebPlanner. The multi-agent design of MindSearch enables the whole framework to seek and integrate information parallelly from larger-scale (e.g., more than 300) web pages in 3 minutes, which is worth 3 hours of human effort. MindSearch demonstrates significant improvement in the response quality in terms of depth and breadth, on both close-set and open-set QA problems. Besides, responses from MindSearch based on InternLM2.5-7B are preferable by humans to ChatGPT-Web and Perplexity.ai applications, which implies that MindSearch can already deliver a competitive solution to the proprietary AI search engine.