Web-Scale Semantic Product Search with Large Language Models (Amazon Science, PAKDD 2023)

Overview

This entry indexes an Amazon Science publication, "Web-scale semantic product search with large language models," presented at PAKDD 2023.

• Source: Amazon Science publication page
• Venue: PAKDD 2023
• Category: Verticals — Industrial blog / publication

Context and Motivation

Semantic product search at web scale is one of the central challenges in large-scale search, recommendation, and personalization systems. Traditional pipelines treat retrieval, ranking, and generation as separate stages, which limits the system's ability to handle natural-language interaction, multi-hop reasoning, and real-time knowledge. This work addresses how large language models can be applied to product search across massive e-commerce catalogs.

Typical Architecture (as outlined in the post)

1. Input and representation — encoding queries, documents, and user context into dense or sparse representations, or structured prompts. 2. Core modules — retriever, reranker, planner, memory, and tool interfaces composed in sequence or parallel. 3. Learning strategies — supervised fine-tuning, contrastive learning, distillation, reinforcement learning, and synthetic data. 4. Inference strategies — single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.

Evaluation Considerations

• Datasets: MS MARCO, BEIR, Natural Questions, proprietary domain corpora, public recommendation datasets.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
• Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.

Specific quantitative results are not reproduced in this post; consult the original publication PDF for exact numbers.

Engineering Trade-offs

| Concern | Guidance | |---------|----------| | Data | PII handling, partitioned indexes, rollback-capable embedding versions | | Latency | p99 budgets, cascaded retrieval with early stopping, query caching, async reranking | | Quality | Interleaving experiments, human audits, citation verification | | Safety | Source whitelisting, adversarial detection, output filtering | | Cost | Small-model routing, distillation, hybrid sparse-dense retrieval |

Key Takeaways for Search / RecSys Practitioners

1. Architecture: cascaded retrieval + reranking + generation remains mainstream; agentic paradigms make retrieval strategy itself learnable. 2. Data: high-quality instruction data and click/session logs are critical; synthetic data needs leakage and distribution-shift safeguards. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Product: latency, cost, explainability, and safety are hard production constraints—do not optimize academic benchmarks alone.

Related Entries

• An interpretable ensemble of graph and language models
• Applying Deep Learning to Ads Conversion Prediction in Last Mile Delivery
• Automated Query-Product Relevance Labeling using Large Language Models
• Behavior Modeling Space Reconstruction for E-Commerce Search
• Behavior-driven query similarity prediction based on pre-trained language models
• Better to Ask in English: Cross-Lingual Evaluation of Large Language Models

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decision-making and tool use | | Gen-IR | Generative Information Retrieval |