Unified Embedding Based Personalized Retrieval in Etsy Search
Overview
- Paper: Unified Embedding Based Personalized Retrieval in Etsy Search
- Authors: Rishikesh Jha, Siddharth Subramaniyam, Ethan Benjamin, Thrivikrama Taula (Etsy)
- Published: 2023-06-07
- Source: https://arxiv.org/abs/2306.04833
- Topic: Personalization in search and recommendation
- Graph embeddings — capturing item-user interaction structure
- Transformer embeddings — semantic text representations
- Term-based embeddings — lexical matching signals
- Feature engineering for personalization
- Hard negative sampling strategies
- Application of transformer models, including a novel pre-training strategy
- Other tricks for improving search relevance and deploying such a model at industry scale
- Original paper: Unified Embedding Based Personalized Retrieval in Etsy Search (arXiv:2306.04833)
TL;DR
Etsy shares an end-to-end trained unified embedding model that solves both the semantic gap on tail queries and the broad-intent problem on popular queries via personalization, deployed at industry scale with significant online gains.
Background and Motivation
Embedding-based neural retrieval is a prevalent approach to address the semantic gap problem that often arises in product search on tail queries. In contrast, popular queries typically lack context and have a broad intent, where additional context from a user's historical interactions can be helpful.
This paper addresses both problems: first the semantic gap, then an end-to-end trained model for personalized semantic retrieval.
Approach
The core idea is a unified embedding model that incorporates:
These are trained end to end, with design choices optimized for the tradeoff between retrieval performance and serving efficiency at scale.
The paper also shares practical learnings on:
Results
The personalized retrieval model significantly improved the overall search experience, measured on live traffic and aggregated across multiple A/B tests:
| Metric | Improvement | |--------|-------------| | Search purchase rate | +5.58% | | Site-wide conversion rate | +2.63% |
Why It Matters
1. Architecture: A unified embedding that fuses graph, transformer, and term signals shows how to combine semantic matching with personalization in a single retriever rather than separate cascaded stages. 2. Data: Hard negative sampling and pre-training strategies are critical for embedding quality; click/session logs provide the personalization signal. 3. Production readiness: The paper emphasizes the performance-efficiency tradeoff and real-world A/B validation, not just offline benchmarks — a useful reference for teams deploying neural retrieval in industry.
Original Abstract (English)
> Embedding-based neural retrieval is a prevalent approach to address the semantic gap problem which often arises in product search on tail queries. In contrast, popular queries typically lack context and have a broad intent where additional context from users historical interaction can be helpful. In this paper, we share our novel approach to address both: the semantic gap problem followed by an end to end trained model for personalized semantic retrieval. We propose learning a unified embedding model incorporating graph, transformer and term-based embeddings end to end and share our design choices for optimal tradeoff between performance and efficiency. We share our learnings in feature engineering, hard negative sampling strategy, and application of transformer model, including a novel pre-training strategy and other tricks for improving search relevance and deploying such a model at industry scale. Our personalized retrieval model significantly improves the overall search experience, as measured by a 5.58% increase in search purchase rate and a 2.63% increase in site-wide conversion rate, aggregated across multiple A/B tests - on live traffic.