Evaluating Large Language Models for Cross-Lingual Retrieval (arXiv 2509.14749)
Paper: Evaluating Large Language Models for Cross-Lingual Retrieval Authors: Longfei Zuo, Pingjun Hong, Oliver Kraus, Barbara Plank, Robert Litschko Venue: arXiv preprint, September 2025 Topic areas: Multilingual IR, embedding models, LLM-based retrieval
> Note: The original forum post is a catalog entry built from metadata; the abstract text and quantitative results are not reproduced in the post itself. Details below reflect the paper's framing and the post's surrounding analysis. Consult the original PDF for exact experimental numbers.
Key points
- The paper presents a systematic evaluation of large language models for cross-lingual retrieval (CLIR), positioning LLM-based retrievers/rerankers against multilingual embedding models in the modern IR stack.
- It belongs to a growing line of work questioning whether general-purpose LLMs can replace or augment dedicated retrieval components when queries and documents span different languages.
- The entry is indexed alongside related work on embedding model evaluation, cross-lingual retrieval of scientific documents (CLIRudit, arXiv:2504.16264), and cross-lingual cross-modal retrieval.
- Gap between offline metrics (nDCG@10, MRR, Recall@k) and online user satisfaction; LLM-as-judge still requires human cross-validation.
- English-centric training data and unknown cross-lingual generalization.
- Latency, cost, and hallucination risks as hard production constraints.
- IR — Information Retrieval
- RAG — Retrieval-Augmented Generation
- nDCG — Normalized Discounted Cumulative Gain, a ranking quality metric
- Agentic Search — Modeling search as sequential decisions and tool calls by an LLM agent
- Gen-IR — Generative Information Retrieval
- A Comprehensive Evaluation of Embedding Models and LLMs for IR and QA
- CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents
- Cl2cm: Improving Cross-Lingual Cross-Modal Retrieval
- Cross-Lingual Cross-Modal Retrieval With Noise-Robust Fine-Tuning (IEEE)
- Multimodal LLM Enhanced Cross-lingual Cross-modal Retrieval, MM 2024
- The Cross-Lingual Cost: Retrieval Biases in RAG over Arabic-English Corpora
Context from the post's analysis
Where this work sits in the IR evolution
Neural IR has progressed from BM25 → BERT cross-encoders → dense bi-encoder retrieval → late interaction → generative retrieval and LLM agents. Each generation trades off effectiveness, efficiency, and maintainability. In the LLM era, new variables include the inference budget and the action space (whether to retrieve, how many times, which tools to call).
Practical checklist for engineering adoption
| Concern | Suggested practice | |---------|--------------------| | Data | PII scrubbing, partitioned indexes, rollback-capable embedding versions | | Latency | Cascaded retrieval + early stopping, query caching, async reranking | | Quality | Interleaving experiments, human audits, citation verification | | Safety | Source allowlists, poisoning detection, output filtering | | Cost | Small-model routing, distillation, hybrid sparse+dense retrieval |