Query2doc: Query Expansion with Large Language Models (arXiv, March 2023)
- Authors / Affiliation: Liang Wang, Nan Yang, Furu Wei (Microsoft Research)
- Published: March 2023
- Source: https://arxiv.org/abs/2303.07678
- Type: Academic paper
- Section: Query Understanding
- Inference-time expansion for sparse retrieval: Generate a pseudo-document with an LLM via few-shot prompting, concatenate it with the original query, and feed the expanded query to BM25. The added terms and phrasings close the lexical gap between query and documents.
- Inference-time expansion for dense retrieval: Concatenate the query, generated pseudo-document, and (for conversational queries) dialogue history before encoding, producing a richer query embedding without fine-tuning the retriever.
- Training-time supervision for dense retrieval: Use pseudo-documents as additional positive supervision when fine-tuning a dense retriever with contrastive learning, treating LLM generations as a form of knowledge distillation; this avoids query-document over-similarity issues and improves out-of-domain performance.
- Datasets: MS MARCO and BEIR benchmarks (including Natural Questions); evaluation under both in-domain and out-of-domain settings.
- Metrics: MRR@10 and nDCG@10.
- Baselines: BM25, strong dense retrievers, HyDE, and conventional query expansion methods.
- Findings: Query2doc consistently outperforms these baselines for both sparse and dense retrieval; the fine-tuned variant with pseudo-documents as supervision yields particularly large gains, especially in out-of-domain (BEIR) transfer. Ablations show results are robust across numbers of generated pseudo-documents and prompting templates.
- Extra LLM inference adds latency and cost per query;
- Generation quality depends on the LLM's knowledge of rare or long-tail topics, risking hallucinated expansion terms;
- Evaluation is primarily on English benchmarks; cross-lingual generalization is not addressed.
- Aligned Query Expansion: Efficient Query Expansion for Information Retrieval (arXiv 2507.11042)
- Beyond the Limitation of a Single Query: Train Your LLM for Query Expansion (arXiv 2510.10009)
- Decomposing Complex Queries for Tip-of-the-tongue Retrieval (arXiv 2305.15053)
- Few-Shot Generative Conversational Query Rewriting (SIGIR 2020)
- Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion (arXiv 2504.14175)
- Liang Wang, Nan Yang, Furu Wei. *Query2doc: Query Expansion with Large Language Models*. arXiv:2303.07678, March 2023. https://arxiv.org/abs/2303.07678
One-line summary
Query2doc expands short queries by prompting large language models to generate pseudo-documents, which are then used (directly or as training supervision) to improve both sparse and dense retrieval.
Background and Motivation
Short, keyword-style queries remain the norm in web and enterprise search, creating a vocabulary and semantic mismatch between queries and relevant documents. Traditional pseudo-relevance feedback (PRF) expands queries using top retrieved passages, but is prone to drift and requires an initial retrieval pass. The authors leverage the world knowledge of instruction-tuned LLMs to synthesize relevant content for a query, effectively asking: *what would a document answering this query look like?*
Core Contributions
Method Pipeline
1. Prompt an instruction-tuned LLM (e.g., GPT-3.5) with a few-shot template containing example query → short answer document pairs;
2. Sample pseudo-documents for the target query;
3. Sparse retrieval: append pseudo-document text to the raw query for BM25;
4. Dense retrieval (zero-shot): encode [query; pseudo-doc] (plus dialogue history if present) with a pre-trained dense encoder;
5. Dense retrieval (fine-tuned): train the encoder so that [query; pseudo-doc] embeddings are close to gold passages, using both in-batch negatives and the expanded representations.
Experiments and Evaluation
Note: exact numbers should be verified against the PDF tables at arXiv:2303.07678.
Key Takeaways for Search / Rec / Personalization
1. LLM-generated pseudo-documents are a cheap, label-free way to inject world knowledge into retrieval at query time; 2. The same generations double as supervision signals, improving dense retrievers even without extra relevance labels; 3. Latency and token cost of generation must be weighed against retrieval gains — caching or smaller LLMs can mitigate this; 4. Related follow-ups (HyDE, multi-query expansion, agentic query rewriting) form a growing family of LLM-based query transformation techniques.