Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Images

Summary

This paper, posted on zhichai.net, introduces Appear2Meaning, a cross-cultural benchmark for inferring structured cultural metadata—such as creator, origin, and period—from images. The task of extracting structured cultural information from visual input remains underexplored. The authors, Yuechen Jiang, Enze Zhang, and Md Mohsinul Kabir, present a multi-category benchmark spanning diverse cultures and evaluate vision-language models (VLMs) using an LLM-as-Judge framework. Results show significant performance variation across cultures and metadata types, indicating that current models produce inconsistent and weakly grounded predictions when reasoning about cultural context. The findings highlight fundamental limitations of state-of-the-art VLMs in structured cultural metadata inference and motivate more culturally robust multimodal evaluation. Available as arXiv preprint 2504.06851 (cs.CV), published April 9, 2025.