Generating Multi-turn Clarification for Web Information Seeking (WWW 2024)

This entry summarizes a forum post indexing the WWW 2024 paper "Generating Multi-turn Clarification for Web Information Seeking".

Key points

• Paper: Generating Multi-turn Clarification for Web Information Seeking, accepted at The ACM Web Conference (WWW) 2024.
• Official link: https://dl.acm.org/doi/abs/10.1145/3589334.3645712
• Topic area: Conversational search and multi-turn interaction — systems that proactively generate clarification questions across multiple dialogue turns to resolve ambiguous or underspecified user information needs in open-domain web search.

Context in modern information retrieval

The post situates the paper in the ongoing shift of search and recommendation systems in the LLM era:

• Traditional pipelines separate retrieval, ranking, and generation, which struggles to meet user expectations for natural-language interaction, multi-hop reasoning, and up-to-date knowledge.
• LLM-era systems treat inference budget and action space (whether to retrieve, how many times, which tools to call) as first-class design variables; agentic search makes retrieval strategy itself a learnable, sequential decision process.
• Evaluation is moving from static ranking metrics (nDCG@10, MRR, Recall@k) toward task success rate, citation accuracy, and multi-turn consistency, though LLM-as-judge still requires cross-validation with human assessment.

Typical method and evaluation setup noted in the post

• Components: query/document encoders, retrievers, rerankers, planners, memory modules, tool interfaces.
• Training strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), synthetic data bootstrapping.
• Benchmarks commonly cited in this area: MS MARCO, BEIR, Natural Questions, plus proprietary conversational logs.
• Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.

Engineering considerations highlighted

| Concern | Suggested practice | |---------|--------------------| | Data | PII masking, partitioned indexes, versioned/rollback-capable embeddings | | Latency | Cascade retrieval with early stopping, query caching, async reranking | | Quality | Interleaving experiments, human audits, citation verification | | Safety | Source whitelisting, poisoning detection, output filtering | | Cost | Small-model routing, distillation, hybrid sparse+dense retrieval |

Limitations and open problems

The post lists recurring open questions for this line of work: evaluation trustworthiness, latency and cost constraints, hallucination and safety risks, and cross-lingual / multimodal generalization. It also cautions that paper abstracts emphasize headline metrics, while engineering teams must additionally weigh index update frequency, embedding version compatibility, online A/B sensitivity, and failure modes (empty retrieval, wrong tool calls, over-generation).

Related entries referenced in the post

• A Survey on Multi-Turn Interaction Capabilities of Large Language Models (arXiv:2501.09959)
• Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey (arXiv:2503.22458)
• Aligning Query Representation with Rewritten Query and Relevance Judgments (ACM DOI 10.1145/3627673.3679534)
• An Empirical Analysis on Multi-turn Conversational Recommender Systems (ACM DOI 10.1145/3626772.3657893)
• Beyond Whole Dialogue Modeling: Contextual Disentanglement for Conversation (arXiv:2504.17427)
• CHIQ: Contextual History Enhancement for Improving Query Rewriting (arXiv:2406.05013)

> Note: Quantitative results should be verified against the original PDF. This post is an editorial index entry rather than the paper itself; the original English abstract was not reproduced in the source.