Two Heads Are Better Than One: Improving Search Effectiveness Through LLM-Generated Query Variants (RMIT University, 2025)

Overview

This forum post summarizes a 2025 academic paper from RMIT University: "Two Heads Are Better Than One: Improving Search Effectiveness Through LLM-Generated Query Variants." The paper falls under the Query Understanding area of information retrieval (IR) research.

• Source (PDF): https://www.damianospina.com/publication/ran-2025-two/ran-2025-two.pdf
• Type: Academic paper
• Category: Query Understanding / Search Effectiveness

Background and Motivation

Large-scale search, recommendation, and personalization systems have long faced challenges around efficiency, scalability, and user intent understanding. Traditional pipeline approaches often treat retrieval, ranking, and generation as disjoint stages, which struggles to meet the demands of the LLM era: natural language interaction, multi-hop reasoning, and up-to-date knowledge.

The core idea of this paper is that a single user query is often an imperfect expression of information need. By generating multiple query variants with an LLM — effectively letting "two heads" (the original query and LLM-generated variants) work together — the system improves the chance of retrieving relevant documents that a single formulation would miss.

Core Contributions

• A unified perspective on using LLM-generated query variants to improve retrieval effectiveness
• A clear decomposition of method components (representation, retriever, reranker, planner, generator, feedback mechanisms) to aid engineering adoption
• Reproducible benchmarking practices and coverage that lower the entry cost for follow-up research
• Discussion of interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration
• Identification of open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension

Typical Method Pipeline

1. Input & representation: encode queries, documents, and user context as dense/sparse representations or structured prompts 2. Core modules: retriever, reranker, planner, memory, and tool interfaces, composed sequentially or in parallel 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, RL (including process rewards), and synthetic data bootstrapping 4. Inference strategies: single-shot retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control

Evaluation Setup (Typical for This Line of Work)

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost
• Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs

Exact numerical results should be verified against the original PDF; this summary is based on the abstract and public metadata.

Key Takeaways for Search / Rec / Personalization

1. Architecture: cascaded retrieval + reranking + generation remains mainstream, but agentic paradigms are making retrieval count and policy themselves learnable 2. Data: high-quality instruction data and click/session logs matter as much as models; synthetic data requires protection against leakage and distribution shift 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human assessment 4. Product: latency, cost, explainability, and safety are hard constraints for industrial deployment — do not optimize academic benchmarks alone

Related Work

• Aligned Query Expansion: Efficient Query Expansion for Information Retrieval
• Beyond the Limitation of a Single Query: Train Your LLM for Query Expansion
• Decomposing Complex Queries for Tip-of-the-tongue Retrieval (2023)
• Few-Shot Generative Conversational Query Rewriting (SIGIR 2020), DOI: 10.1145/3397271.3401323
• Hierarchical Query Classification in E-commerce Search (WWW 2024)
• Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion (2025)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Modeling search as sequential decision-making and tool use | | Gen-IR | Generative Information Retrieval |

Who Should Read This

• Researchers: reproduce core comparisons; check whether statistical significance and compute cost are reported
• Engineers: evaluate pluggable modules (encoder, reranker, planner) for integration cost with existing stacks
• Product managers: focus on user-perceivable benefits (latency, answer trustworthiness, multi-turn consistency) rather than offline nDCG alone