Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations (Google, RecSys 2019)
- Source: Google Research publication page
- Venue: RecSys 2019
- Authors: Xinyang Yi, Ji Yang, Aravindan Raghuveer, Baotao Cai, Lichan Hong, Ed H. Chi (Google)
- Two-tower retrieval architecture: User and item representations are learned independently, enabling scalable ANN-based retrieval over very large item corpora instead of exhaustive scoring.
- In-batch softmax training: Negative items are reused from the other examples in the batch, drastically improving training efficiency compared with uniform negative sampling per example.
- Sampling-bias correction (core contribution): Streaming training data comes from click logs, so in-batch items follow the *item popularity* distribution rather than a uniform one. The paper applies a logQ correction — subtracting the log of each item's estimated sampling frequency from its logit — to remove this popularity bias in the softmax objective.
- Streaming-friendly estimation: The paper shows how the sampling probability of each item can be estimated incrementally during streaming training with constant memory, making the correction practical in industrial online-learning settings.
- Feature ablations: Experiments study the impact of different item-side features on retrieval quality.
- Live experiments: The model was evaluated on YouTube recommendations, showing significant improvements in engagement metrics (e.g., watch time) in online A/B tests, beyond offline metric gains.
- Dot-product interaction between towers limits expressive power compared with cross-encoder style scoring; it trades ranking precision for retrieval scalability.
- Popularity frequency estimation may lag under rapidly shifting content distributions.
- Evaluation details and production constraints are described only at the level allowed by the paper's disclosure.
- Original paper: Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations (RecSys 2019)
- 360Brew: A Decoder-only Foundation Model for Personalized Ranking
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers
- Augmenting Netflix Search with In-Session Adapted Recommendations
- Bridging Language and Items for Retrieval and Recommendation
- Data-efficient Fine-tuning for LLM-based Recommendation (SIGIR 2024)
- DiffKG: Knowledge Graph Diffusion Model for Recommendation (WSDM 2024)
Overview
This Google paper addresses candidate retrieval in large-scale recommendation systems, where the system must select a small set of relevant items from a corpus of millions or billions. It proposes a two-tower neural model: a user tower encodes user features and context, while an item tower encodes item features. The two embeddings are compared via dot product, which allows pre-computing item embeddings and serving retrieval through approximate nearest-neighbor search.
Key Points
Why It Matters
The logQ-corrected in-batch softmax became a standard recipe for two-tower retrieval models across industry (YouTube, Google Play, and many later systems). It resolves the tension between training efficiency (sharing negatives within a batch) and correctness (unbiased learning of user-item relevance), and demonstrates that retrieval-stage model quality directly translates into measurable online gains.