[论文] GenOlympiad: Benchmarking Frontier Image Models at Their Capability Bo...

论文概要

研究领域: CV
作者: Xi Hu, Martin Bell, Liang Zheng
发布时间: 2026-10-19
arXiv: 2610.26850

中文摘要

文生图模型已取得巨大进展，但在任何学科中都会被问到的一个问题是：前沿模型在哪些方面不足？我们引入GenOlympiad基准测试，从九个能力轴提供最具挑战性的文本提示来压力测试前沿文生图模型。评估能力包括生成大量对象、深度推理、硬物理一致性、复杂规则遵循等。提示设计使得对齐度可视觉验证（如计数）而非依赖主观判断。我们开发自动化评估框架，将提示分解为简单和困难问题进行评分。评估了20多个模型包括Nano Banana Pro、GPT-Image-2.5、Seedream-5.0等。即使最佳模型（Nano Banana Pro）严格评分也仅获23.9/100分，而GPT-6 Astra等编码模型具有更高的提示对齐分数但视觉美感不如专门模型。

原文摘要

Text-to-image models have made tremendous progress and can produce high-quality images. However, a question which would be asked in any other discipline is where the frontier models fall short. We introduce GenOlympiad, a benchmark which provides the most challenging text prompts from nine capability axes for stress-testing frontier text-to-image models. The assessed capabilities include generating large quantities, performing deep reasoning, ensuring hard physical consistency, following complex rules and so on. The prompts are designed such that prompt alignment is visually verifiable (e.g., counting) instead of relying on subjectiveness (e.g., style). Based on this, we develop an automated evaluation framework which decomposes the prompt into easy and hard questions, and for most hard qu...

自动采集于 2026-10-21

#论文 #arXiv #CV #小凯