CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer (Alibaba, AAAI 2024)

Overview

This entry covers the AAAI 2024 paper CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer from Alibaba.

• Venue: AAAI 2024
• Affiliation: Alibaba
• Official link: https://ojs.aaai.org/index.php/AAAI/article/view/28376
• Topic area: Multi Lingual / cross-lingual cross-modal retrieval

Problem Setting

Cross-lingual cross-modal retrieval aims to match queries in one language against content (typically images or multimodal documents) described in another. In practice, large-scale vision-language pretraining data is heavily skewed toward English, leaving target languages under-served. CL2CM tackles this gap by transferring cross-modal alignment knowledge from a well-resourced source language to target languages, improving retrieval quality where target-language training pairs are scarce or noisy.

Key Points

• Proposes a cross-lingual knowledge transfer approach for cross-modal retrieval (CL2CM).
• Targets the common real-world scenario where vision-language pretraining is English-centric while deployment must serve other languages.
• Published at AAAI 2024, a top-tier AI conference.

Context and Related Work

This paper sits within a broader research thread on multilingual and multimodal retrieval, including:

• Cross-lingual cross-modal retrieval with noise-robust fine-tuning
• Multimodal LLM enhanced cross-lingual cross-modal retrieval (ACM MM 2024)
• Evaluation studies of LLMs for cross-lingual retrieval

Note

This forum post is based on the paper's title and public metadata. For the full method details, architecture, and quantitative benchmarks, please refer to the original AAAI proceedings page.