The Cross-Lingual Cost: Retrieval Biases in RAG over Arabic-English Corpora
Source: arXiv:2507.07543 (July 2025)
Authors: Chen Amiraz, Yaroslav Fyodorov, Elad Haramaty, Zohar Karnin, Liane Lewin-Eytan
Overview
This paper investigates retrieval biases in retrieval-augmented generation (RAG) systems operating over bilingual Arabic-English corpora. As RAG deployments increasingly serve multilingual user bases, the language composition of the knowledge base and the language of the query can interact in unexpected ways, affecting which documents get retrieved and, ultimately, answer quality.
Key Points
- The paper studies a cross-lingual cost: when a RAG system operates over a mixed Arabic-English corpus, retrieval quality can suffer depending on the language relationship between the query and the underlying evidence.
- The core issue is language-matching bias in retrieval: retrievers tend to favor documents in the same language as the query, so when the best supporting evidence is written in the other language, it may be under-retrieved — even if the underlying generator (LLM) is fully capable of using it.
- This isolates the retriever, not the generator, as the bottleneck in cross-lingual RAG: the generation component can often produce good answers in either language, provided the right context is retrieved.
- The study uses Arabic-English corpora as the testbed, a pairing that is practically relevant for Middle East deployments and representative of asymmetric-resource language settings.
- The work has practical implications for multilingual RAG system design and evaluation: teams should evaluate retrieval in cross-lingual conditions (query language ≠ evidence language) rather than only in matched-language settings, and consider language-aware retrieval strategies when indexing bilingual knowledge bases.
- Paper: The Cross-Lingual Cost: Retrieval Biases in RAG over Arabic-English Corpora, arXiv, July 2025.
Why It Matters
Most RAG benchmarks are English-centric or evaluate retrieval with queries and documents in the same language. This paper highlights a failure mode that such evaluations miss: systems can look healthy in monolingual tests yet systematically under-retrieve the most relevant evidence in real bilingual deployments. For organizations serving Arabic-speaking users with mixed-language knowledge bases, this is a direct production concern.