Hybrid Hierarchical Retrieval for Open-Domain Question Answering (ACL 2023 Findings)
This entry is a curated listing from a zhichai.net awesome-list covering hybrid search vs. vector search. It references the paper:
- Title: Hybrid Hierarchical Retrieval for Open-Domain Question Answering
- Venue: ACL 2023 (Findings), July 2023
- Link: https://aclanthology.org/2023.findings-acl.679/
- Type: Academic paper
- A unified perspective for comparing scattered work in the hybrid-retrieval space
- A clear decomposition of method components: representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms
- Reproducible benchmarks/datasets/classification tables that lower the entry barrier for follow-up research
- Discussion of interfaces with LLM tool calling, reinforcement learning, and multi-agent collaboration
- Explicit open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, task success rate, latency, token cost
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs
- Azure AI Search: Outperforming vector search with hybrid retrieval
- COS-Mix: Cosine Similarity and Distance Fusion for Improved Information Retrieval (arXiv 2406.00638)
- Deep Retrieval at CheckThat! 2025 (arXiv 2505.23250)
- Domain-specific Question Answering with Hybrid Search (arXiv 2412.03736)
- Efficient Knowledge Graph Construction and Retrieval (arXiv 2507.03226)
- Modernizing Facebook Scoped Search: Keyword and Embedding Hybrid Retrieval (arXiv 2509.13603)
> Note on fidelity: The forum post is largely a standardized template entry. Specific experimental numbers are not reproduced here; readers should consult the original PDF before citing quantitative results.
Context and Motivation
The post situates the paper within the broader challenge of large-scale search, recommendation, and personalization systems: balancing efficiency, scalability, and user-intent understanding. Traditional pipelines separate retrieval, ranking, and generation, which struggles to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and real-time knowledge. Hybrid hierarchical retrieval addresses open-domain QA by combining retrieval strategies at multiple levels.
Core Contributions (as outlined in the entry)
Typical Method Pipeline
The entry describes a standard four-step pattern:
1. Input & representation — encode queries, documents, and user context into dense/sparse representations or structured prompts 2. Core modules — retrievers, rerankers, planners, memory modules, and tool interfaces composed in series or parallel 3. Learning strategies — supervised fine-tuning, contrastive learning, distillation, RL (including process rewards), bootstrapped data synthesis 4. Inference strategy — single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control
Evaluation Conventions Referenced
Engineering Checklist (from the post's appendix)
| Item | Question | Recommendation | |------|----------|----------------| | Data | PII in training/index? Version management? | Partitioned indexes, anonymization, rollback-safe embedding versions | | Latency | p99 budget? Retrieval steps? | Cascades + early stopping, query caching, async reranking | | Quality | Do offline gains translate online? | Interleaving experiments, human audits, citation verification | | Safety | Poisoning/bias via open retrieval? | Source allowlists, adversarial detection, output filtering | | Cost | Tokens & GPU per query? | Model routing, distillation, hybrid sparse+dense |