Behavior-Driven Query Similarity Prediction Based on Pre-Trained Language Models for E-commerce Search
Source: Amazon Science — SIGIR 2023 eCommerce workshop Link: https://www.amazon.science/publications/behavior-driven-query-similarity-prediction-based-on-pre-trained-language-models-for-e-commerce-search
Summary
This Amazon Science work, presented at the SIGIR 2023 eCommerce workshop, tackles query similarity prediction for e-commerce search by combining pre-trained language models (PLMs) with user behavior signals.
Why it matters
Query similarity underpins several downstream search functions:
- Query rewriting and expansion
- Related-query / query suggestion
- Retrieval matching between customer queries and product listings
- Neural IR has evolved from BM25 → BERT cross-encoders → bi-encoder dense retrieval → generative retrieval and LLM-based methods.
- Dense retrieval is efficient via approximate nearest-neighbor search but sensitive to domain shift and long-tail queries.
- Behavior signals (clicks, sessions, purchases) act as weak supervision that can calibrate embeddings toward actual user intent.
- An interpretable ensemble of graph and language models for improving search relevance
- Applying Deep Learning to Ads Conversion Prediction in Last Mile Delivery
- Automated Query-Product Relevance Labeling using Large Language Models
- Behavior Modeling Space Reconstruction for E-Commerce Search (arXiv 2501.18216)
- Better to Ask in English: Cross-Lingual Evaluation of Large Language Models
- Beyond Relevance: A Demand Balancer Model for Rental Platforms
Purely textual similarity often fails in e-commerce: short, ambiguous queries (e.g., brand abbreviations, colloquial product names) may look dissimilar as strings yet correspond to identical shopping intent. Behavior data — such as co-click and session patterns — provides ground-truth evidence of which queries users treat as interchangeable.
Approach direction
The work leverages PLM representations enriched with behavior-driven supervision, so that learned query embeddings reflect real customer intent rather than surface form. This follows the broader industry trend of coupling neural language understanding with implicit feedback from search logs.
Context in the IR / Search Landscape
Engineering Considerations
| Aspect | Consideration | |--------|---------------| | Data | Behavior logs must be cleaned, de-identified, and versioned; beware popularity bias | | Latency | Bi-encoder / embedding-based similarity enables pre-computation and ANN lookup | | Evaluation | Combine offline metrics (Recall@k, nDCG) with online A/B tests on CTR and satisfaction | | Cold start | New queries lack behavior; PLM priors help bootstrap embeddings |
Note on Quantitative Results
Specific experimental numbers should be verified against the original publication PDF via the Amazon Science link above. This page primarily provides metadata, positioning, and cross-references.
Related Entries
Glossary
| Term | Meaning | |------|---------| | IR | Information Retrieval | | PLM | Pre-Trained Language Model | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | ANN | Approximate Nearest Neighbor search | | Co-click | Two items/queries clicked within the same user session, used as behavioral similarity signal |