Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling

This forum post discusses the April 2025 arXiv paper *Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling* (arXiv: 2504.05216), authored by Hengran Zhang, Keping Bi, Jiafeng Guo, Xiaojie Sun, Shihao Liu, Daiting Shi and others (8 authors total). It is categorized under Query Understanding in the forum's awesome list.

Background and Motivation

Information retrieval in large-scale search, recommendation, and personalization systems has long faced challenges in efficiency, scalability, and user intent understanding. Traditional pipelines separate retrieval, ranking, and generation, making it hard to meet LLM-era demands for natural language interaction, multi-hop reasoning, and up-to-date knowledge. This paper addresses how LLMs can be better leveraged for dense retrieval, drawing on the query likelihood modeling principle from classical probabilistic IR.

Core Contributions

• Proposes a unified perspective that brings related scattered work into a comparable framework
• Provides a clear decomposition of method components: representation learning, retriever, reranker, planner, generator, and feedback mechanisms
• Offers reproducible benchmarks, datasets, or classification tables to lower the entry cost for follow-up research
• Discusses interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, outlining a path from research prototypes to industrial systems
• Lists open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension

Method / System Architecture

The typical workflow follows four steps: problem formalization → model/system design → training or construction → inference pipeline.

1. Input & representation: encode queries, documents, and user context as dense or sparse representations, or build structured prompts 2. Core modules: retriever, reranker, planner, memory modules, tool interfaces — chained or parallel 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), bootstrapped data synthesis 4. Inference strategies: single-turn retrieval, iterative retrieval, parallel sub-queries, early stopping and budget control

Experiments and Evaluation

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost
• Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs
• Ablations: contribution of retrieval steps, reranking depth, and training data scale

Specific numerical results should be verified against the original paper's tables; this post is based on the abstract and public metadata.

Key Insights for Search / Rec / Personalization

1. Architecture: cascaded retrieval + rerank + generation remains mainstream, but the agentic paradigm makes "number and strategy of retrievals" itself a learnable object 2. Data: high-quality instruction data and click/session logs are both critical; synthetic data must guard against knowledge leakage and distribution shift 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation 4. Product: latency, cost, explainability, and safety policies are hard constraints for industrial deployment — do not optimize academic benchmarks alone

Limitations and Future Work

Possible limitations include experiment scale constrained by GPU budgets, benchmark–real-user distribution mismatch, English-centric data with unknown cross-lingual generalization, and safety risks of agentic systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommender systems.

Related Entries

• Aligned Query Expansion: Efficient Query Expansion for Information Retrieval (arXiv 2507.11042)
• Beyond the Limitation of a Single Query: Train Your LLM for Query Expansion (arXiv 2510.10009)
• Decomposing Complex Queries for Tip-of-the-tongue Retrieval (arXiv 2305.15053)
• Few-Shot Generative Conversational Query Rewriting (SIGIR 2020)
• Hierarchical Query Classification in E-commerce Search (WWW 2024)
• Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion (arXiv 2504.14175)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |

Practical Checklist for Engineers

| Item | Question | Suggestion | |------|----------|------------| | Data | PII in training/index? Version management? | Partitioned indexes, anonymization, rollback-safe embedding versions | | Latency | p99 budget? Retrieval steps? | Cascade + early stopping, cache hot queries, async rerank | | Quality | Do offline gains translate to online CTR? | Interleaving experiments, human audits, citation checks | | Safety | Poisoning/bias from open retrieval? | Source whitelists, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Route to small models, distillation, hybrid sparse+dense |

Source: arXiv:2504.05216