Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling (arXiv 2504.05216)

Forum topic · 小凯 · 2026-07-05

Summary

This arXiv paper (April 2025, arXiv:2504.05216), authored by Hengran Zhang, Keping Bi, Jiafeng Guo, Xiaojie Sun, Shihao Liu, Daiting Shi and colleagues, explores how to better leverage large language models (LLMs) for dense retrieval by modeling query likelihood. It belongs to the query understanding area of information retrieval research, addressing how retrieval, ranking, and generation can be re-organized in the LLM era to handle natural language interaction, multi-hop reasoning, and real-time knowledge needs. The surrounding discussion frames the work within the evolution of neural IR—from BM25 and bi-encoder dense retrieval to cross-encoders, late interaction, generative retrieval, and agentic search—highlighting trade-offs among efficiency, effectiveness, and maintainability. It also covers practical engineering considerations for deployment, including latency budgets, index versioning, safety concerns such as corpus poisoning and hallucination, and cost control via model routing and distillation. Evaluation typically involves benchmarks like MS MARCO, BEIR, and Natural Questions with metrics such as nDCG@10, MRR, and Recall@k, though exact quantitative results should be verified against the original PDF. Open problems include evaluation trustworthiness, cross-lingual generalization, and multimodal extension.

Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling (April 2025, arXiv)

Metadata

| Field | Content | |-------|---------| | Title | Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling | | Authors / Affiliations | Hengran Zhang, Keping Bi, Jiafeng Guo, Xiaojie Sun, Shihao Liu, Daiting Shi, et al. (8 authors total) | | Published | April 2025, arXiv | | Link | https://arxiv.org/abs/2504.05216 | | Category | Query Understanding |

Overview

This paper investigates how to unlock the potential of large language models (LLMs) in dense retrieval through query likelihood modeling. It sits at the intersection of information retrieval and large-scale search/recommendation systems, responding to the core question of how to redistribute responsibilities among retrieval, ranking, generation, and tool calling in the LLM era.

Context and Motivation

Large-scale search, recommendation, and personalization systems have long faced challenges in efficiency, scalability, and user intent understanding. Traditional pipeline approaches treat retrieval, ranking, and generation as separate stages, which struggles to meet LLM-era demands for natural language interaction, multi-hop reasoning, and real-time knowledge. This work aims to advance the theoretical and practical boundaries of this intersection.

Positioning in the Field

Neural information retrieval has evolved through several generations, each balancing efficiency–effectiveness–maintainability:

BM25: sparse lexical matching, strong baseline
Bi-encoder dense retrieval: millisecond-level recall via approximate nearest neighbor search, but sensitive to domain shift and long-tail queries
Cross-encoders: high accuracy but document representations cannot be precomputed
Late interaction: compromise between the two
Generative retrieval and LLM agents: reduce cascade errors but face index-update difficulties

RAG and agentic search extend external knowledge access from one-shot retrieval to an iterative, verifiable, plannable process, shifting evaluation from static nDCG toward task success rate, citation accuracy, and multi-hop reasoning chain completeness.

Typical Methodology (for this class of work)

1. Input and representation: encode queries, documents, and user context into dense/sparse representations or structured prompts 2. Core modules: retriever, reranker, planner, memory, tool interfaces 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), bootstrapped data synthesis 4. Inference strategies: single-round retrieval, iterative retrieval, parallel sub-queries, early stopping and budget control

Evaluation

Datasets: MS MARCO, BEIR, Natural Questions, domain corpora
Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost
Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs

Exact numerical results should be verified against the original PDF.

Engineering Deployment Checklist

| Item | Question | Recommendation | |------|----------|----------------| | Data | Does training/indexing contain PII? How is versioning managed? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascade + early stopping, cache hot queries, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, manual audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query token and GPU usage? | Route to smaller models, distillation, hybrid sparse+dense |

Key Takeaways

1. Architecture: cascade retrieval + reranking + generation remains mainstream, but agentic paradigms are making retrieval count and strategy themselves learnable 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation 4. Product: latency, cost, interpretability, and safety are hard constraints for industrial deployment—academic benchmarks alone are insufficient

Related Entries

Aligned Query Expansion: Efficient Query Expansion for Information Retrieval (arXiv 2507.11042)
Beyond the Limitation of a Single Query: Train Your LLM for Query Expansion (arXiv 2510.10009)
Decomposing Complex Queries for Tip-of-the-tongue Retrieval (arXiv 2305.15053)
Few-Shot Generative Conversational Query Rewriting (SIGIR 2020, DOI: 10.1145/3397271.3401323)
Hierarchical Query Classification in E-commerce Search (WWW 2024)
Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion (arXiv 2504.14175)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Modeling search as sequential decision-making and tool calling | | Gen-IR | Generative Information Retrieval |