MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents (EMNLP 2021)

Overview

MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents was published at EMNLP 2021 (ACL Anthology). The paper addresses a key limitation of prior document-grounded dialogue work: most existing datasets ground each dialogue in a single document, while real-world information-seeking conversations often require consulting multiple related documents.

• Source: https://aclanthology.org/2021.emnlp-main.498/
• Venue: EMNLP 2021, ACL
• Category: Evaluation of Search Engines / Document-Grounded Dialogue

Key Contributions

• New task definition: goal-oriented dialogue modeling where responses must be grounded in multiple documents from a document collection, rather than a single fixed document.
• MultiDoc2Dial dataset: a large-scale dataset of goal-oriented dialogues grounded in multiple documents. The dialogues are built on four thematic document collections, each containing a set of related documents (drawn from US federal student aid information), with each dialogue turn annotated with grounding evidence.
• Baseline models: the authors propose and evaluate several baseline approaches, including retrieval-based methods and end-to-end models, on tasks such as response generation and grounding (evidence) prediction.
• Benchmark findings: experimental results show that grounding responses across multiple documents is significantly more challenging than single-document grounding, establishing a strong baseline landscape for future work.

Relevance to Search / IR / RAG

This work sits at the intersection of:

• Conversational search: users formulate information needs across dialogue turns, and the system must locate relevant evidence in a corpus.
• Retrieval-augmented generation (RAG): the task anticipates multi-hop, multi-document grounding, a core challenge for modern RAG and agentic search systems.
• Evaluation: MultiDoc2Dial provides a reproducible benchmark for measuring how well systems can retrieve and use evidence from multiple documents during dialogue.

Related Entries

• A Dataset of Information-Seeking Questions and Answers Anchored in Research Abstracts
• Evaluation of Retrieval-Augmented Generation: A Survey (May 2024)
• ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation
• AI Search Has A Citation Problem (Mar 2025, CJR)

References

• Original paper: MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents, EMNLP 2021. https://aclanthology.org/2021.emnlp-main.498/

> Note: Quantitative results should be verified against the original PDF; this page is based on the paper's public metadata and abstract.