XRAG: Cross-lingual Retrieval-Augmented Generation (Amazon, May 2025)
Paper: https://arxiv.org/abs/2505.10089 Authors: Wei Liu, Sony Trenous, Leonardo F. R. Ribeiro, Bill Byrne, Felix Hieber Affiliations: Amazon; Heidelberg Date: May 2025 (arXiv) Topic area: Multilingual / Cross-lingual RAG
Overview
XRAG studies cross-lingual Retrieval-Augmented Generation: how RAG pipelines behave when the query language, the retrieved documents, and the expected answer language do not all match. The work sits at the intersection of RAG and large-scale search/recommendation systems, where traditional pipelines that treat retrieval, ranking, and generation as separate stages struggle to meet LLM-era requirements such as natural language interaction, multi-hop reasoning, and up-to-date external knowledge.
Positioning in the RAG landscape
- Architecture: The dominant pattern remains cascade retrieval + rerank + generate, but agentic paradigms are increasingly making the number and strategy of retrieval steps themselves learnable.
- Components: Typical decompositions include dense/sparse encoders, retrievers, rerankers, planners, memory modules, and tool interfaces, combined in series or parallel.
- Training strategies: Supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), and synthetic data bootstrapping.
- Inference strategies: Single-shot retrieval, iterative retrieval, parallel sub-queries, early stopping, and compute budget control.
- A Comprehensive Evaluation of Embedding Models and LLMs for IR and QA
- CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents (arXiv:2504.16264)
- CL2CM: Improving Cross-lingual Cross-modal Retrieval
- Cross-Lingual Cross-Modal Retrieval With Noise-Robust Fine-Tuning (IEEE)
- Evaluating Large Language Models for Cross-Lingual Retrieval (arXiv:2509.14749)
- Multimodal LLM Enhanced Cross-lingual Cross-modal Retrieval (MM 2024)
- XRAG: Cross-lingual Retrieval-Augmented Generation. arXiv, May 2025. https://arxiv.org/abs/2505.10089
Evaluation context
Common benchmarks and metrics in this space include MS MARCO, BEIR, and Natural Questions, measured with nDCG@10, MRR, Recall@k, task success rate, and human preference—against baselines such as BM25, dense retrieval, cross-encoder rerankers, and retrieval-free LLMs. Specific quantitative results for XRAG should be taken from the original PDF rather than secondary summaries.
Key takeaways for practitioners
1. Architecture: Agentic search reframes retrieval as sequential decision-making; retrieval count and policy become optimization targets. 2. Data: High-quality instruction data and session logs remain critical; synthetic data requires leakage and distribution-shift checks. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human assessment. 4. Deployment: Latency, cost, explainability, and safety are hard constraints in production—do not optimize academic benchmarks alone.
Limitations and open questions
The broader area faces challenges including benchmark-vs-real-user distribution mismatch, English-centric data limiting cross-lingual generalization, hallucination and security risks in open-web retrieval, and cost of test-time compute. Future directions include better test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommendation.