XOR QA: Cross-lingual Open-Retrieval Question Answering
Source: https://arxiv.org/abs/2010.11856 (arXiv, October 2020)
Authors / Affiliations: Akari Asai, Jungo Kasai, Jonathan H. Clark, Kenton Lee, Eunsol Choi, Hannaneh Hajishirzi
Category: Evaluation of Search Engines
Key Points
- Introduces cross-lingual open-retrieval question answering (XOR QA): questions are asked in non-English languages, answers must be retrieved from English Wikipedia, and returned in the questioner's language.
- This task setting reflects a realistic scenario: the bulk of high-quality knowledge on the web is in English, while users search in their native languages.
- The benchmark covers multiple answer types: single-span, multiple-span, and yes/no questions.
- The paper contributes a task definition, dataset, and baseline systems combining cross-lingual representation learning with open-domain retrieval and answer generation.
- Sits at the intersection of information retrieval and multilingual NLP, anticipating later work on retrieval-augmented generation (RAG) and multilingual search evaluation.
- The source post is largely a metadata/template wrapper around the paper; the detailed background, method, and experiment sections in the post are generic commentary rather than the paper's content.
- Specific experimental numbers were not provided in the source; consult the original PDF for quantitative results and exact experimental protocols.
- Researchers: Use XOR QA as a benchmark for cross-lingual retrieval + generation pipelines; report statistical significance and compute costs.
- Engineers: Treat retriever, reranker, and generator as pluggable modules; cross-lingual retrieval from an English corpus is a reusable pattern for multilingual products.
- Product managers: Multilingual answer quality depends on retrieval coverage across languages, not just on monolingual ranking metrics.
- Evaluation of Retrieval-Augmented Generation: A Survey (arXiv:2405.07437)
- A Dataset of Information-Seeking Questions and Answers Anchored in Research Articles (arXiv:2105.03011)
- ARES: An Automated Evaluation Framework for RAG (arXiv:2311.09476)
- AI Search Has A Citation Problem (CJR, March 2025)
Context in the Field
The work addresses long-standing challenges in large-scale search and information retrieval: cross-lingual access to knowledge, user intent understanding across languages, and the pipeline separation of retrieval, ranking, and generation. It is best viewed alongside related research on open-domain QA datasets, RAG evaluation, and multilingual benchmarks (e.g., the related items in this collection such as evaluation surveys for RAG and anchored QA datasets).