WikiVQABench: A Human-Curated Knowledge-Grounded Visual Question Answering Benchmark

Paper Overview

Field: Computer Vision Authors: Basel Shbita, Pengyuan Li, Anna Lisa Gentile Published: 2025-05-20 arXiv: 2505.15981

Summary

Visual Question Answering (VQA) benchmarks have largely emphasized perception-based tasks that can be solved from visual content alone. In contrast, many real-world scenarios require external knowledge that is not directly observable in the image to answer correctly.

The authors introduce WikiVQABench, a human-curated, knowledge-grounded VQA benchmark constructed by systematically combining:

• Wikipedia images
• Their associated article captions
• Structured knowledge from Wikidata

Pipeline

1. Large language models (LLMs) generate candidate multiple-choice image-question-answer sets. 2. All generated instances are subsequently reviewed and curated by human annotators to ensure:

• Factual correctness
• Visual-text consistency
• That each question requires external knowledge in addition to visual evidence

Evaluation Results

The benchmark includes a large collection of Wikipedia images and curated multiple-choice questions, designed to evaluate knowledge-aware vision-language models (VLMs). Evaluation of 15 VLMs ranging from 256M to 90B parameters shows a wide performance spread:

• Accuracy range: 24.7% to 75.6%

This demonstrates that the benchmark effectively discriminates models on knowledge-intensive reasoning capabilities.

Availability

The dataset and benchmark code are publicly available via the paper page: https://arxiv.org/abs/2505.15981

--- *Auto-collected on 2026-05-22*