Semantic Ads Retrieval at Walmart eCommerce with Language Models Progressively Trained on Multiple Knowledge Domains
This post summarizes an arXiv paper on semantic ads retrieval deployed in the context of Walmart eCommerce, using language models progressively trained on multiple knowledge domains.
Paper metadata
- Title: Semantic Ads Retrieval at Walmart eCommerce with Language Models Progressively Trained on Multiple Knowledge Domains
- Authors: Zhaodong Wang, Weizhi Du, Md Omar Faruk Rokon, Pooshpendu Adhikary, Yanbing Xue, Jiaxuan Xu, et al. (9 authors in total)
- Published: March 2025 (arXiv, listed as 2502.09089)
- Source: https://arxiv.org/abs/2502.09089
- Category: Verticals / Industrial IR
- From BM25 to BERT cross-encoders, dense bi-encoder retrieval, late interaction, and finally generative retrieval and LLM-based agentic search.
- Dense retrieval enables millisecond-level recall via approximate nearest neighbor search but is sensitive to domain shift and long-tail queries.
- Cross-encoders offer high precision but cannot precompute document representations.
- Generative approaches reduce cascade errors but face index-update difficulties.
- An interpretable ensemble of graph and language models for improving search
- Applying Deep Learning to Ads Conversion Prediction in Last Mile Delivery (arXiv 2502.10514)
- Automated Query-Product Relevance Labeling using Large Language Models (arXiv 2502.15990)
- Behavior Modeling Space Reconstruction for E-Commerce Search (arXiv 2501.18216)
- Behavior-driven query similarity prediction based on pre-trained language models
- Better to Ask in English: Cross-Lingual Evaluation of Large Language Models (DOI: 10.1145/3589334.3645643)
One-line summary
The paper presents Walmart's approach to semantic ads retrieval, where language models are progressively trained across multiple knowledge domains to improve query–ad understanding at eCommerce scale.
Background and motivation
In large-scale search, recommendation, and personalization systems, information retrieval has long faced challenges around efficiency, scalability, and user intent understanding. Traditional pipelines often separate retrieval, ranking, and generation, which struggles to meet the combined demands of natural language interaction, multi-hop reasoning, and up-to-date knowledge in the LLM era. This paper is situated in that context, targeting the ads retrieval vertical of eCommerce search.
Context within neural IR
The post situates the work within the broader evolution of neural information retrieval:
For recommendation, the core tensions remain sparse user behavior, massive item catalogs, and multi-objective business trade-offs. LLMs provide semantic priors and cold-start capability, but online inference cost and hallucination risk demand careful system design.
Engineering checklist for production deployment
| Aspect | Question | Suggestion | |--------|----------|------------| | Data | Does training/indexing contain PII? How are versions managed? | Partitioned indexes, anonymization, rollback-safe embedding versions | | Latency | What is the p99 budget? How many retrieval hops? | Cascaded retrieval + early stopping, caching popular queries, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, manual audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Route to smaller models, distillation, hybrid sparse+dense retrieval |
Takeaways for readers
1. Researchers: Reproduce the core comparisons; check whether the paper reports statistical significance and compute costs. 2. Engineers: Extract pluggable modules (encoder, reranker, planner) and evaluate integration costs against existing stacks. 3. Product managers: Focus on user-perceivable benefits — latency, answer trustworthiness, multi-turn consistency — rather than offline nDCG alone.