Web-Scale Semantic Product Search with Large Language Models (Amazon Science, PAKDD 2023)
Overview
This entry summarizes the Amazon Science publication "Web-scale semantic product search with large language models", presented at PAKDD 2023.
- Source: Amazon Science publication page
- Type: Industrial publication / conference paper (PAKDD 2023)
- Domain: E-commerce search, information retrieval, large language models
- Semantic understanding of queries: Moving beyond keyword matching toward intent-level matching between natural language queries and product catalogs.
- LLM integration into search stacks: How LLM-based representations and generation complement or replace classic components (BM25, dense retrievers, cross-encoder rerankers).
- Web-scale constraints: Latency budgets, indexing costs, and scalability requirements that distinguish industrial systems from research prototypes.
- Evaluation: The widening gap between offline ranking metrics (nDCG, MRR, Recall@k) and online user satisfaction.
- An interpretable ensemble of graph and language models for improving search relevance
- Applying deep learning to ads conversion prediction in last-mile delivery
- Automated query-product relevance labeling using large language models
- Behavior modeling space reconstruction for e-commerce search
- Behavior-driven query similarity prediction based on pre-trained language models
- Original publication: Web-scale semantic product search with large language models — Amazon Science, PAKDD 2023.
Background and Motivation
Web-scale product search faces persistent challenges in information retrieval: efficiency, scalability, and understanding user intent. Traditional pipeline-based approaches separate retrieval, ranking, and generation, making it difficult to meet modern expectations for natural language interaction and semantic understanding. This work addresses how large language models can be applied to semantic product search at web scale, re-examining the division of responsibilities between retrieval, ranking, and generation in the LLM era.
Key Themes
Engineering Considerations
For teams deploying LLM-enhanced product search, the following trade-offs are central:
| Concern | Consideration | |---------|---------------| | Latency | p99 budget; number of retrieval steps; cascading with early stopping | | Quality | Whether offline gains translate to online CTR/satisfaction | | Cost | Per-query token and GPU cost; model routing and distillation | | Safety | Hallucination, poisoned or biased retrieved content, output filtering | | Freshness | Index update frequency and embedding version compatibility |
Takeaways for Search / Recommendation Practitioners
1. Architecture: Cascaded retrieval + rerank + generation remains the mainstream; agentic paradigms are making retrieval strategy itself learnable. 2. Data: High-quality instruction data and click/session logs matter as much as model architecture. 3. Evaluation: LLM-as-judge should be cross-validated with human assessment. 4. Product: Latency, cost, interpretability, and safety are hard constraints in production, not just academic benchmarks.