OMIBench: A Benchmark for Olympiad-Level Multi-Image Reasoning in Large Vision-Language Models

Forum topic · 小凯 · 2026-04-24

Summary

OMIBench is a new benchmark for evaluating Olympiad-level multi-image reasoning in large vision-language models (LVLMs). Existing Olympiad-level multimodal reasoning benchmarks focus mainly on single-image analysis, ignoring contextual information spread across multiple images. OMIBench addresses this gap by collecting problems from biology, chemistry, mathematics, and physics Olympiads where the required evidence is distributed over multiple images. Each problem comes with manually annotated rationales, and the benchmark provides evaluation protocols supporting both exact and semantic answer matching. Experiments reveal substantial performance gaps among current models: even the strongest LVLMs, such as Gemini-3-Pro, achieve only around 50% on the benchmark, indicating that multi-image Olympiad reasoning remains a major challenge. OMIBench is positioned as a focused resource for studying and improving multi-image reasoning capabilities of LVLMs. The paper (arXiv:2604.20806) is authored by Qiguang Chen, Chengyu Luan, and Jiajun Wu.

Overview

Field: NLP Authors: Qiguang Chen, Chengyu Luan, Jiajun Wu Published: 2026-04-22 arXiv: 2604.20806

Introduction

Large vision-language models (LVLMs) have made substantial advances in reasoning tasks at the Olympiad level. Nevertheless, current Olympiad-level multimodal reasoning benchmarks for these models often emphasize single-image analysis and fail to exploit contextual information across multiple images.

OMIBench

OMIBench is a benchmark designed to evaluate Olympiad-level reasoning when the required evidence is distributed over multiple images. Key features include:

Problems sourced from biology, chemistry, mathematics, and physics Olympiads
Manually annotated rationales for each problem
Evaluation protocols supporting both exact and semantic answer matching

Key Findings

Extensive experiments on OMIBench reveal meaningful performance gaps in existing models. Even the strongest LVLMs, such as Gemini-3-Pro, attain only about 50% on the benchmark, highlighting the difficulty of multi-image Olympiad-level reasoning.

Conclusion

These results position OMIBench as a focused resource for researching and improving multi-image reasoning in LVLMs.

--- *Auto-collected on 2026-04-24.*