Summary
A new paper on arXiv (2504.06851) by Yuechen Jiang, Enze Zhang, and Md Mohsinul Kabir introduces Appear2Meaning, a multi-category, cross-cultural benchmark for inferring structured cultural metadata—such as creator, origin, and period—from visual inputs. The task remains underexplored in computer vision, and the authors evaluate vision-language models (VLMs) using an LLM-as-Judge framework. Results show significant performance variation across different cultures and metadata types, with models producing inconsistent and weakly grounded predictions. The findings highlight the limitations of current VLMs in structured cultural metadata inference and underscore the need for culturally robust benchmarks and evaluation methods. This is an English translation/summary of a Chinese forum post from zhichai.net covering the paper's abstract, released on arXiv on April 9, 2025, in the cs.CV category.
Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Images
Field: cs.CV
Authors: Yuechen Jiang, Enze Zhang, Md Mohsinul Kabir
Published: 2025-04-09
arXiv: 2504.06851
Abstract (translated)
Inferring structured cultural metadata (e.g., creator, origin, period) from visual inputs remains underexplored. This paper introduces a multi-category, cross-cultural benchmark for this task and evaluates vision-language models (VLMs) using an LLM-as-Judge framework. The results show that models exhibit significant performance variation across different cultures and metadata types, leading to inconsistent and weakly grounded predictions. These findings highlight the limitations of current VLMs in structured cultural metadata inference.
Key points
- Introduces a multi-category, cross-cultural benchmark for cultural metadata inference from images.
- Targets structured metadata fields such as creator, origin, and period.
- Uses an LLM-as-Judge framework for evaluation.
- Reveals significant performance variation across cultures and metadata types.
- Shows that current VLM predictions can be inconsistent and weakly grounded.
---
*Auto-collected on 2025-04-10*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177169720