Hierarchical Query Classification in E-commerce Search (WWW 2024)

Overview

This entry tracks the Amazon Science publication "Hierarchical query classification in e-commerce search", presented at The Web Conference (WWW) 2024.

• Source: https://www.amazon.science/publications/hierarchical-query-classification-in-e-commerce-search
• Type: Industry publication (WWW 2024)
• Topic area: Query Understanding, Verticals

Why It Matters

In large-scale search, recommendation, and personalization systems, query classification is a foundational component of query understanding. Classifying e-commerce queries into a hierarchical taxonomy (rather than a flat category set) allows the system to respect the structure of the product catalog, improving retrieval coverage, ranking relevance, and vertical routing decisions.

Problem Setting

The work sits at the intersection of information retrieval and industrial-scale search. Classical pipelines treat recall, ranking, and generation as separate stages; the LLM era adds new variables such as inference budget and action space (whether to retrieve, how many times, which tools to call). Hierarchical query classification helps disambiguate user intent before these downstream stages run.

Typical System Pipeline

The generic pipeline pattern relevant to this line of work:

1. Input & representation: encode queries, documents, and user context into dense/sparse representations or structured prompts; 2. Core modules: retriever, reranker, planner, memory modules, tool interfaces — chained or parallelized per task; 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning, synthetic data bootstrapping; 4. Inference strategies: single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping and budget control.

Evaluation Context

Common datasets and metrics in this space include MS MARCO, BEIR, Natural Questions, and domain-specific corpora; metrics such as nDCG@10, MRR, Recall@k, latency, and human preference. Baselines typically include BM25, dense retrieval, cross-encoder rerankers, and retrieval-free LLMs. Specific quantitative results for this paper should be verified against the original PDF, as this post summarizes design logic from abstract-level metadata.

Engineering Checklist

| Concern | Question | Suggestion | |---------|----------|------------| | Data | PII in training/index? Version control? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | p99 budget? Retrieval steps? | Cascaded retrieval + early stop, hot-query caching, async reranking | | Quality | Do offline gains translate online? | Interleaving experiments, human audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelisting, adversarial detection, output filtering | | Cost | Per-query token/GPU cost? | Small-model routing, distillation, hybrid sparse+dense |

Takeaways for Search / Rec / Personalization

1. Architecture: cascade retrieval + rerank + generation remains mainstream, but agentic paradigms are making retrieval strategy itself learnable; 2. Data: high-quality instruction data and click/session logs remain critical; synthetic data needs leakage and distribution-shift safeguards; 3. Evaluation: offline metrics and online satisfaction are diverging; LLM-as-judge needs cross-validation with human evaluation; 4. Product: latency, cost, explainability, and safety are hard industrial constraints — optimize beyond academic benchmarks.

Related Entries

• Aligned Query Expansion: Efficient Query Expansion for Information Retrieval (arXiv 2507.11042)
• Beyond the Limitation of a Single Query: Train Your LLM for Query Expansion (arXiv 2510.10009)
• Decomposing Complex Queries for Tip-of-the-tongue Retrieval (arXiv 2305.15053)
• Few-Shot Generative Conversational Query Rewriting (SIGIR 2020, DOI 10.1145/3397271.3401323)
• Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion (arXiv 2504.14175)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making with tool calls | | Gen-IR | Generative Information Retrieval |

References

• Original publication: Hierarchical query classification in e-commerce search, WWW 2024.