Scaling Laws for Online Advertisement Retrieval (arXiv 2411.13322)

Overview

This entry covers an academic paper posted to arXiv in November 2024:

• Title: Scaling Laws for Online Advertisement Retrieval
• Link: https://arxiv.org/abs/2411.13322
• Authors: Yunli Wang, Zhen Zhang, Zixuan Yang, Tianyu Xu, Zhiqiang Wang, Yu Li, et al. (12 authors total)
• Type: Research paper (Verticals section of the list)

Context

In large-scale search, recommendation, and personalization systems, information retrieval has long faced challenges around efficiency, scalability, and user intent understanding. Traditional pipeline approaches tend to treat retrieval, ranking, and generation as separate stages. This paper studies scaling laws — how retrieval performance changes as model size, data, and compute are scaled — specifically in the context of online advertisement retrieval, a setting with strict latency and cost constraints.

Positioning within the field

The work sits at the intersection of information retrieval and industrial-scale ad systems. From a systems perspective, neural retrieval has evolved through several generations:

• Sparse retrieval (e.g., BM25)
• Dense dual-tower retrieval with approximate nearest-neighbor search
• Cross-encoder rerankers (BERT-era)
• Late-interaction models, generative retrieval, and LLM-based agentic search

Each generation trades off efficiency, effectiveness, and maintainability. Scaling-law analysis adds a quantitative dimension: how much does retrieval quality improve per unit of additional model capacity or training data, and where do diminishing returns set off?

Key points

• The paper addresses resource allocation questions for ad retrieval models: how to choose model size, training data scale, and compute budget.
• Online advertising imposes hard constraints — p99 latency, per-query cost, and business metrics (CTR, conversion) — so scaling decisions must balance academic benchmarks against production realities.
• The entry recommends cross-checking quantitative claims against the original PDF, since exact experimental tables are not reproduced here.

Engineering checklist for retrieval systems

| Concern | Question | Suggestion | |---------|----------|------------| | Data | Does training/indexing data contain PII? How is versioning handled? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascaded retrieval with early stopping, hot-query caching, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelisting, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Small-model routing, distillation, hybrid sparse+dense retrieval |

Related entries

• Applying Deep Learning to Ads Conversion Prediction in Last Mile Delivery
• Automated Query-Product Relevance Labeling using Large Language Models
• Behavior Modeling Space Reconstruction for E-Commerce Search, Jan 2025
• Behavior-driven query similarity prediction based on pre-trained language models

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Search modeled as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |

> Note: This summary is based on public metadata and abstract-level information. For exact experimental results, refer to the original paper at https://arxiv.org/abs/2411.13322.