LLM-QE: Aligning Large Language Models with Ranking Preferences for Better Query Expansion

Forum topic · 小凯 · 2026-07-05

Summary

LLM-QE is a February 2025 arXiv paper (arXiv:2502.17057) by Sijia Yao, Pengcheng Huang, Zhenghao Liu, Yu Gu, Yukun Yan, Shi Yu and colleagues that improves query expansion for information retrieval by aligning large language models with ranking preferences. Instead of relying solely on supervised fine-tuning, the method trains LLMs to generate expanded queries that directly improve downstream ranking quality, using preference-based optimization signals derived from ranking outcomes. The work addresses a known gap where LLM-generated query expansions are fluent but do not necessarily retrieve better documents. Reported experiments show that aligning expansion generation with ranking preferences yields stronger retrieval performance than conventional expansion baselines on standard benchmarks. This article summarizes the paper's motivation, method, experimental design, and implications for search and recommendation systems, with cross-references to related work on query expansion, conversational query rewriting, and LLM-based retrieval augmentation.

LLM-QE: Improving Query Expansion by Aligning Large Language Models with Ranking Preferences

Paper: LLM-QE: Improving Query Expansion by Aligning Large Language Models with Ranking Preferences (arXiv, February 2025)
Authors: Sijia Yao, Pengcheng Huang, Zhenghao Liu, Yu Gu, Yukun Yan, Shi Yu, et al. (7 authors total)
Category: Query Understanding

One-sentence summary

LLM-QE improves LLM-based query expansion by optimizing the expansion model against downstream ranking preferences, so that generated expansions actually improve retrieval quality rather than merely reading fluently.

Background and motivation

Query expansion is a long-standing technique in information retrieval: augmenting a user's short or ambiguous query with additional terms or generated content can improve recall. With large language models (LLMs), expansions can now be generated as natural-language or hypothetical documents. However, a common failure mode is that LLM-generated expansions — while linguistically plausible — do not necessarily rank relevant documents higher, because the generation objective is not tied to the retrieval/ranking objective.

LLM-QE addresses this mismatch by aligning the expansion model with ranking preferences: preference signals come from which expansions lead to better ranking outcomes, rather than from imitation of reference expansions alone.

Core contributions

Frames query expansion as a preference-alignment problem rather than a pure generation problem.
Uses ranking outcomes as the supervision/preference signal for optimizing the LLM expansion policy.
Demonstrates improved retrieval performance over standard query-expansion baselines on public benchmarks (see original paper tables for exact numbers).
Situates the work within the broader line of LLM-for-IR research: hypothetical documents, query rewriting, and retrieval-augmented generation.

Method outline

1. Input: an original user query. 2. Expansion generation: the LLM produces expanded queries / pseudo-documents conditioned on the query. 3. Ranking feedback: candidate expansions are evaluated by their effect on document ranking quality. 4. Preference optimization: the model is updated to prefer expansions that yield better rankings (in the spirit of preference-based fine-tuning such as DPO-style objectives). 5. Inference: at retrieval time, the aligned model expands queries, which are then fed to a retriever/ranker.

Experimental design

Tasks: open-domain text retrieval with LLM-based query expansion.
Baselines: classical expansion methods, dense retrievers, and prior LLM-based expansion approaches (e.g., HyDE-style hypothetical documents, supervised query expansion).
Metrics: standard IR ranking metrics such as nDCG@k / Recall@k / MRR (consult the PDF for the exact per-dataset numbers).

Note: this post is based on the abstract and public metadata; verify quantitative claims against the original PDF before citing.

Key takeaways for search/RecSys practitioners

1. Objective alignment matters: aligning generation with downstream ranking quality is more effective than optimizing generation quality in isolation. 2. Preference optimization extends beyond chat: ranking preferences are a viable reward signal for IR-side LLMs. 3. Cost/latency: expansion adds inference cost per query; consider caching and cascades in production.

Related entries

Aligned Query Expansion: Efficient Query Expansion for Information Retrieval (arXiv 2507.11042)
Beyond the limitation of a single query: Train your LLM for query expansion (arXiv 2510.10009)
Decomposing Complex Queries for Tip-of-the-tongue Retrieval (arXiv 2305.15053)
Few-Shot Generative Conversational Query Rewriting, SIGIR 2020 (DOI: 10.1145/3397271.3401323)
Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion (arXiv 2504.14175)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | Query expansion | Augmenting a query with additional terms/documents to improve retrieval | | DPO / preference optimization | Fine-tuning models on preferred vs. dispreferred outputs | | nDCG | Normalized Discounted Cumulative Gain, a ranking-quality metric |