Query Rewriting in Retrieval-Augmented Large Language Models (EMNLP 2023)

Forum topic · 小凯 · 2026-07-05

Summary

This forum post introduces the EMNLP 2023 paper "Query Rewriting in Retrieval-Augmented Large Language Models", published in the ACL Anthology (2023.emnlp-main.322). The paper addresses a core weakness of retrieval-augmented generation (RAG): user queries are often unanswerable by retrieval alone, requiring reformulation before search. The work proposes rewriting the original query before it is passed to the retrieval module, so that the retriever returns passages that better support answering the user's actual intent. The post situates the paper within the broader query understanding landscape, alongside related work on aligned query expansion, LLM-trained query expansion, complex query decomposition, and few-shot generative conversational query rewriting (SIGIR 2020). It also provides practical engineering commentary: cascaded retrieval plus reranking remains mainstream, evaluation gaps between offline metrics and online satisfaction are widening, and latency, cost, safety, and citation accuracy constrain production deployment. A glossary of terms (RAG, nDCG, agentic search, generative IR) and a checklist for data, latency, quality, security, and cost are included for practitioners. Note that quantitative results should be verified against the original PDF, as the post is partly based on abstracts and public metadata.

Query Rewriting in Retrieval-Augmented Large Language Models (EMNLP 2023)

Source: ACL Anthology — 2023.emnlp-main.322

Key points

The paper targets a known failure mode of retrieval-augmented generation (RAG): the original user query is frequently not directly retrievable against — the retrieved passages fail to answer the question even though the knowledge exists in the corpus.
The proposed approach rewrites the query before it reaches the retriever, reformulating it into a form whose retrieval results are more likely to contain answer-supporting evidence. A reranking step selects the most relevant passages before generation.
The work belongs to the broader query understanding line of research, alongside:
Aligned Query Expansion (arXiv 2507.11042)
Training LLMs for query expansion beyond a single query (arXiv 2510.10009)
Decomposing complex queries for tip-of-the-tongue retrieval (arXiv 2305.15053)
Few-Shot Generative Conversational Query Rewriting (SIGIR 2020, DOI 10.1145/3397271.3401323)
Hierarchical query classification in e-commerce search (WWW 2024)
Rethinking LLM-based query generation and knowledge leakage (arXiv 2504.14175)

Methodology context

Typical pipelines in this problem space follow four stages: problem formalization → model/system design → training or construction → inference pipeline.

1. Input and representation — encode queries, documents, and user context into dense/sparse representations or structured prompts; 2. Core modules — retriever, reranker, planner, memory, tool interfaces, chained or parallel per task; 3. Learning strategies — supervised fine-tuning, contrastive learning, distillation, reinforcement learning, synthetic data bootstrapping; 4. Inference strategies — single-shot retrieval, iterative retrieval, parallel sub-queries, early stopping and budget control.

Evaluation context

Standard evaluation in this area uses datasets such as MS MARCO, BEIR, and Natural Questions, with metrics like nDCG@10, MRR, Recall@k, Hit@k, and task success rate, against baselines including BM25, dense retrieval, cross-encoder reranking, and non-retrieval LLMs. Exact quantitative results should be verified against the original PDF.

Insights for search and recommendation practitioners

1. Architecture: cascaded retrieval + reranking + generation remains mainstream, but agentic paradigms increasingly treat *retrieval policy itself* as learnable. 2. Data: high-quality instruction data and click/session logs matter as much as models; synthetic data risks leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human assessment. 4. Productization: latency, cost, explainability, and safety are hard constraints — do not optimize academic benchmarks alone.

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decisions and tool calls | | Gen-IR | Generative Information Retrieval |

Query Rewriting in Retrieval-Augmented Large Language Models (EMNLP 2023)

Summary

Query Rewriting in Retrieval-Augmented Large Language Models (EMNLP 2023)

Key points

Methodology context

Evaluation context

Insights for search and recommendation practitioners

Glossary

Tags