MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents (EMNLP 2021)

This forum entry is an index/metadata page for the EMNLP 2021 ACL paper MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents.

• Paper page: <https://aclanthology.org/2021.emnlp-main.498/>
• Venue: EMNLP 2021 (ACL Anthology, main conference)
• Category in the source list: Evaluation of Search engines

What the paper is about

MultiDoc2Dial addresses goal-oriented dialogue grounded in multiple documents instead of a single document or passage. The work introduces:

• A new task formulation for multi-document-grounded dialogue;
• A corresponding dataset built on public customer service documents;
• Baseline models for the task.

Note: the original forum post's own abstract fields are placeholder text, so specific quantitative results should be verified against the PDF at the ACL Anthology link above.

Context provided by the post

The surrounding Chinese commentary is largely a reusable template about modern information retrieval and retrieval-augmented systems. Its useful points:

• Classic search stacks follow a funnel of recall → ranking → generation; LLM-era systems add reasoning budget and action space (whether/when to retrieve, which tools to call).
• Neural IR has evolved from BM25 to dense retrieval, cross-encoder reranking, and generative/agentic approaches, trading off efficiency, effectiveness, and maintainability.
• Evaluation is shifting from static metrics (nDCG, MRR) toward task success, citation accuracy, and multi-hop reasoning chain quality.

Engineering checklist (from the post)

| Concern | Suggestion | |---|---| | Data privacy | PII scrubbing, partitioned indexes, versioned/rollback-safe embeddings | | Latency | Cascaded retrieval with early stopping, query caching, async reranking | | Quality | Interleaving experiments, human audits, citation verification | | Safety | Source whitelisting, poisoning/adversary detection, output filtering | | Cost | Model routing, distillation, hybrid sparse+dense retrieval |

Related entries cross-referenced in the post

• Evaluation of Retrieval-Augmented Generation: A Survey (arXiv 2405.07437)
• A Dataset of Information-Seeking Questions and Answers Anchored in Research Abstracts
• ARES: An Automated Evaluation Framework for RAG (arXiv 2311.09476)
• AI Search Has A Citation Problem (CJR, Mar 2025)
• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents (arXiv 2401.13178)

Reader guidance from the post

1. Researchers: reproduce core comparisons; check for statistical significance and compute cost reporting. 2. Engineers: evaluate pluggable modules (encoders, rerankers, planners) for integration cost. 3. Product managers: focus on user-perceivable benefits (latency, answer trustworthiness, multi-turn consistency) rather than offline nDCG alone.