Zero-shot Document Retrieval with Hybrid Pseudo-document Retriever, ICASSP 2025
Metadata
| Field | Value | |------|------| | Title | Zero-shot Document Retrieval with Hybrid Pseudo-document Retriever, ICASSP 2025 | | Authors/Affiliation | See source metadata | | Publication | ICASSP 2025 (IEEE) | | Source | https://ieeexplore.ieee.org/abstract/document/10889897 | | Resource type | Academic paper |
TL;DR
The paper addresses zero-shot document retrieval by introducing a hybrid pseudo-document retriever that combines multiple retrieval signals to perform effectively without labeled training data on the target domain.
Background and Motivation
Large-scale search, recommendation, and personalization systems have long faced challenges in efficiency, scalability, and user-intent understanding. Traditional pipeline approaches separate retrieval, ranking, and generation, which is increasingly mismatched with the needs of the large language model (LLM) era: natural-language interaction, multi-hop reasoning, and real-time knowledge grounding.
Zero-shot document retrieval is particularly relevant for:
- Open-domain information access
- Enterprise knowledge retrieval
- Conversational search
- Semantic understanding in recommender systems
- End-to-end architectures that couple external knowledge sources with generative models
- Proposes a unified perspective that places related work within a comparable framework.
- Provides clear decomposition of method components (representation learning, retriever, reranker, planner, generator, feedback mechanism) for engineering deployment.
- Offers reproducible benchmarks, datasets, or taxonomy tables that lower the entry barrier for follow-up researchers.
- Discusses interfaces with LLM tool calling, reinforcement learning, and multi-agent collaboration, mapping a path from research prototype to industrial system.
- Lists open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.
- Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora, public recommendation sets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLM, commercial search APIs.
- Ablations: validating each module (number of retrieval steps, reranking depth, training-data scale) on final quality.
- A Comprehensive Study of Knowledge Editing for Large Language Models
- INTERS: Unlocking the Power of Large Language Models in Search
- Recommendation as Instruction Following: A Large Language Model Empowered Approach
- RouteLLM: Learning to Route LLMs with Preference Data
- Translational Generative Retrieval via Potential Query Generation
- Representation Learning with Large Language Models for Recommendation
- Original paper: *Zero-shot Document Retrieval with Hybrid Pseudo-document Retriever*, ICASSP 2025. See IEEE Xplore.
Core Contributions
Method / System Architecture
The work typically follows a four-step pattern: problem formalization → model/system design → training or construction pipeline → inference pipeline.
1. Input and representation: encode queries, documents, and user context as dense or sparse vectors, or construct structured prompts. 2. Core modules: retriever, reranker, planner, memory module, and tool interface, arranged serially or in parallel according to the task. 3. Learning strategy: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), bootstrapped data synthesis. 4. Inference strategy: single-turn retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.
The hybrid pseudo-document retriever combines multiple retrieval signals (for example sparse BM25 plus dense embeddings, or multiple dense encoders) to construct pseudo-document representations that generalize to unseen domains without supervision.
Experiments and Evaluation
Typical experimental design covers:
Quantitative results should be cross-checked against the original PDF tables; this summary is based on the abstract and public metadata.
Key Findings and Insights
Implications for search, recommendation, and personalization:
1. Architecture: cascade retrieval + rerank + generation remains dominant, but agentic paradigms are turning "how many times and how" to retrieve into a learnable decision. 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data requires protection against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge must be cross-validated with human evaluation. 4. Product: latency, cost, explainability, and safety policies are hard industrial constraints and cannot be ignored in favor of academic benchmarks.
Limitations and Future Work
Limitations may include GPU-budget constraints on experiment scale, mismatches between benchmarks and real user distributions, English-centric data with unknown cross-lingual generalization, and safety risks for agentic systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal and fairness constraints for recommender systems.
Engineering Deployment Checklist
| Item | Question | Recommendation | |------|----------|----------------| | Data | Does the index contain PII? How are versions managed? | Partitioned index, de-identification, rollback-friendly embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascade + early stop, cache hot queries, asynchronous reranking | | Quality | Does offline gain translate to online CTR/satisfaction? | Interleaving, human audit samples, citation validation | | Safety | Does open retrieval introduce poisoning or bias? | Source whitelist, adversarial detection, output filtering | | Cost | Tokens per query and GPU footprint? | Model routing, distillation, sparse+dense hybrid |
Glossary
| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decision-making and tool calling | | Gen-IR | Generative Information Retrieval |
Action Items for Readers
1. Researchers: reproduce core comparative experiments and check for statistical significance and compute cost. 2. Engineers: extract pluggable modules (encoder, reranker, planner) and evaluate integration cost with existing stacks. 3. Product managers: identify user-perceivable benefits (latency, answer credibility, multi-turn consistency) rather than offline nDCG alone.