DISKCO: Disentangling Knowledge from Cross-Encoder to Bi-Encoder (WWW 2024)
Overview
This entry catalogues the paper DISKCO: Disentangling Knowledge from Cross-Encoder to Bi-Encoder, presented at The Web Conference (WWW) 2024 and listed on Amazon Science:
- Source page: <https://www.amazon.science/publications/diskco-disentangling-knowledge-from-cross-encoder-to-bi-encoder>
- Category: Ranking for Search
- Cross-encoders (e.g., BERT-style rerankers) jointly encode the query–document pair, achieving strong relevance judgments but requiring expensive per-pair inference and preventing precomputed document representations.
- Bi-encoders (dual-tower dense retrieval) encode queries and documents independently, enabling approximate nearest-neighbor search at scale, but often trail cross-encoders in ranking quality.
- Dense retrieval achieves millisecond-level recall via ANN search but is sensitive to domain shift and long-tail queries.
- Cross-encoders are accurate but cannot precompute document representations.
- Generative approaches reduce cascade errors but complicate index updates.
- Deep Learning to Rank in Industrial Search Engines and Recommender Systems
- Multi-Objective Recommendation in the Era of Generative AI: A Survey
- A Generative Re-ranking Model for List-level Multi-objective Optimization
- A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE
- Accelerating Listwise Reranking: Reproducing and Enhancing FIRST (SIGIR)
- Adaptive Neural Ranking Framework: Toward Maximized Business Goals
- Original paper: DISKCO: Disentangling Knowledge from Cross-Encoder to Bi-Encoder, WWW 2024. See the Amazon Science publication page and the paper PDF for exact experimental results.
Problem and Motivation
Large-scale search, recommendation, and personalization systems have long faced trade-offs among efficiency, scalability, and intent understanding. Two dominant neural ranking paradigms sit at opposite ends of the efficiency–effectiveness spectrum:
DISKCO targets this gap by disentangling and transferring knowledge from a cross-encoder teacher into a bi-encoder student, aiming for cross-encoder-level quality with bi-encoder-level inference cost.
Where It Sits in the Field
Neural information retrieval has evolved from BM25 to BERT cross-encoder rerankers, dense dual-tower retrieval, late-interaction models, and most recently generative retrieval and LLM-based agentic search. Each generation balances effectiveness, efficiency, and maintainability:
DISKCO belongs to the distillation/knowledge-transfer line of work that tries to get the best of both encoder families within a classic recall → rerank → serve cascade.
Engineering Checklist for Ranking Systems
| Concern | Question | Suggestion | |---|---|---| | Data | Does training/index data contain PII? Versioning? | Partitioned indexes, anonymization, rollback-capable embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascade + early stopping, cache hot queries, async reranking | | Quality | Do offline gains convert to online CTR/satisfaction? | Interleaving experiments, human audits, citation checks | | Safety | Does open retrieval introduce poisoning/bias? | Source allowlists, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Small-model routing, distillation, hybrid sparse+dense retrieval |
Takeaways
1. Architecture: cascade retrieval + rerank + generation remains mainstream, while agentic paradigms increasingly treat "when and how much to retrieve" as a learnable decision. 2. Data: high-quality instruction data and click/session logs matter as much as model design; synthetic data risks leakage and distribution shift. 3. Evaluation: the gap between offline metrics (nDCG@10, MRR) and online satisfaction is widening; LLM-as-judge needs cross-validation against human assessment. 4. Production: latency, cost, interpretability, and safety are hard constraints — optimizing academic benchmarks alone is not enough.