GenOlympiad: A Stress-Test Benchmark for Frontier Text-to-Image Models Across Nine Capability Axes

Forum topic · 小凯 · 2026-10-21

Summary

GenOlympiad is a new benchmark introduced by Xi Hu, Martin Bell, and Liang Zheng (arXiv:2610.26850) designed to stress-test frontier text-to-image models through highly challenging text prompts drawn from nine capability axes. The assessed capabilities include generating large quantities of objects, deep reasoning, hard physical consistency, and following complex rules. Unlike subjective evaluation criteria such as style, the prompts are engineered so that prompt alignment is visually verifiable — for example, by counting objects in the generated image. The authors also develop an automated evaluation framework that decomposes each prompt into easy and hard questions for scoring. The benchmark evaluated more than 20 models, including Nano Banana Pro, GPT-Image-2.5, and Seedream-5.0. Results show even the best model, Nano Banana Pro, achieved only 23.9/100 under strict scoring, while coding-oriented models like GPT-6 Astra achieved higher prompt alignment but with inferior visual aesthetics compared to specialized image models. The benchmark highlights significant gaps in current text-to-image systems on compositional, reasoning, and rule-following tasks.

Paper Overview

Research Area: Computer Vision (CV) Authors: Xi Hu, Martin Bell, Liang Zheng Release Date: 2026-10-19 arXiv: 2610.26850

Abstract

Text-to-image models have made tremendous progress and can produce high-quality images. However, a question which would be asked in any other discipline is where the frontier models fall short.

The authors introduce GenOlympiad, a benchmark which provides the most challenging text prompts from nine capability axes for stress-testing frontier text-to-image models. The assessed capabilities include:

Generating large quantities of objects
Performing deep reasoning
Ensuring hard physical consistency
Following complex rules
And more

The prompts are designed such that prompt alignment is visually verifiable (e.g., counting) instead of relying on subjectiveness (e.g., style). Based on this, the authors develop an automated evaluation framework which decomposes the prompt into easy and hard questions, scoring each component.

Key Findings

20+ models were evaluated, including Nano Banana Pro, GPT-Image-2.5, and Seedream-5.0.
Even the best-performing model, Nano Banana Pro, achieved only 23.9/100 under strict scoring.
Coding-oriented models such as GPT-6 Astra achieved higher prompt alignment scores, but their visual aesthetics fell behind dedicated image generation models.

These results demonstrate that despite rapid progress, frontier text-to-image models still have substantial room for improvement on compositional, reasoning-heavy, and physically consistent generation tasks.

---

*Auto-collected on 2026-10-21*