LLM-based Long-tail Query Rewriting in Taobao Search (WWW 2024)

Overview

This forum post catalogs the paper "Large Language Model based Long-tail Query Rewriting in Taobao Search", presented in the industry track of WWW 2024, by the Taobao Search team at Alibaba.

• Source: https://dl.acm.org/doi/abs/10.1145/3589335.3648298
• Topic area: Query Understanding / Query Rewriting for e-commerce search

Why Long-tail Query Rewriting Matters

In large-scale e-commerce search, most queries are long-tail: they appear rarely or never in training data, so click and purchase behavioral signals are sparse. Traditional supervised query rewriting models trained on behavioral logs degrade badly on such queries, hurting retrieval relevance and downstream conversion. Rewriting a rare query like a niche, colloquial, or multi-intent product description into a semantically clear target query is therefore a core challenge for Taobao search.

Key points

• Multi-instruction-following LLM rewriter: instead of training separate models per rewriting task (intent rewriting, disambiguation, subcategory prediction, knowledge expansion, named-entity recognition), the work casts rewriting as multi-instruction following so one LLM handles diverse rewriting subtasks jointly.
• Chain-of-thought reasoning: the LLM reasons about query intent step by step, producing more accurate rewrites than direct generation, particularly for long-tail queries where pattern matching fails.
• Bidirectional distillation: to meet strict latency and cost budgets of online search, LLM rewriting knowledge is distilled bidirectionally into lightweight models — combining zero-shot generalization of the LLM with the efficiency of small serving models.
• Industrial deployment: the rewriter was deployed in Taobao search and validated via online A/B tests, with reported improvements in long-tail traffic satisfaction and a measurable GMV (Gross Merchandise Volume) gain.

Method Pipeline (as described in the paper)

1. Formulate long-tail query rewriting as instruction-following with task-specific prompts. 2. Apply LLM inference with chain-of-thought to produce the rewritten target query. 3. Distill the LLM rewriter into compact models for online serving under latency constraints. 4. Feed rewrites into the retrieval/relevance stack and evaluate via offline metrics and online A/B experiments.

Context and Related Work

The post cross-references related literature on query rewriting and expansion:

• Aligned Query Expansion for IR (arXiv:2507.11042)
• Training LLMs for query expansion beyond a single query (arXiv:2510.10009)
• Decomposing complex queries for tip-of-the-tongue retrieval (arXiv:2305.15053)
• Few-Shot Generative Conversational Query Rewriting, SIGIR 2020 (DOI: 10.1145/3397271.3401323)
• Hierarchical query classification in e-commerce search, WWW 2024
• Rethinking LLM-based query rewriting: hypothetical documents or knowledge leakage? (arXiv:2504.14175)

Takeaways

• LLMs provide semantic priors that compensate for missing behavioral signals on long-tail queries.
• Distillation is the practical bridge between LLM capability and strict online serving budgets in production search.
• Deployment decisions should weigh latency, cost, and hallucination risk, not just offline rewriting quality; quantitative results should be verified against the original PDF.

Note: This post is largely an index/summary entry; for exact experimental numbers and system details, consult the original paper via the ACM DL link above.