CPI-Bench: A Comprehensive, Practical and Intelligent Benchmark for Real-World Image Editing

Forum topic · 小凯 · 2026-08-18

Summary

CPI-Bench (arXiv:2508.08546) is a new benchmark for evaluating image editing models in real-world scenarios, proposed by Qinye Zhou, Jun Zheng, and Yongchao Du. Existing benchmarks are limited to simple single-image tasks, offer narrow coverage, and fail to distinguish model performance. CPI-Bench addresses this with three subsets: CPI-General-Bench covers diverse editing tasks and pioneers multi-image editing evaluation; CPI-Practical-Bench focuses on high-frequency real user scenarios; CPI-Intelligent-Bench tests reasoning-demanding editing abilities. Evaluations of mainstream image editing models show the benchmark improves discrimination between models and quantifies gaps in general editing, practical deployment, and advanced reasoning editing. Notably, rank correlation analysis shows CPI-Bench aligns best with the Arena image editing leaderboard, indicating it faithfully captures human evaluators' preferences and serves as a strong proxy for real user experience.

Overview

Field: Computer Vision
Authors: Qinye Zhou, Jun Zheng, Yongchao Du
Published: 2026-08-17
arXiv: 2508.08546

Abstract (translated)

With the rapid advancement of image editing models and their widespread application across various domains, there is an increasingly urgent need to deploy these model capabilities directly into real-world scenarios. However, existing benchmarks remain confined to simple single-image tasks, suffering from limited coverage dimensions and an inability to effectively differentiate performance among diverse models. Consequently, they fail to reliably evaluate model performance in complex multi-image editing, highly demanding reasoning instructions, and practical deployment settings.

To address these limitations, the authors propose CPI-Bench, a Comprehensive, Practical and Intelligent benchmark for real-world image editing, comprising three core subsets:

CPI-General-Bench: comprehensively covers diverse editing tasks and pioneers the evaluation of multi-image editing.
CPI-Practical-Bench: focuses on high-frequency, real-world user application scenarios.
CPI-Intelligent-Bench: specifically evaluates editing capabilities that require demanding reasoning.

Key Findings

Evaluations of mainstream image editing models on CPI-Bench show enhanced performance discrimination between models.
The benchmark provides comprehensive and reliable quantification of gaps in general editing ability, practical deployment effectiveness, and advanced reasoning-based editing.
Rank correlation analysis shows CPI-Bench aligns best with the Arena image editing leaderboard, indicating it faithfully captures human evaluators' preferences and perceptual judgments, and serves as a strong proxy for real user experience.

Original Abstract (excerpt)

> With the rapid advancement of image editing models and their widespread application across various domains, there is an increasingly urgent need to deploy these model capabilities directly into real-world scenarios. However, existing benchmarks remain confined to simple single-image tasks, suffering from limited coverage dimensions and an inability to effectively differentiate performance among diverse models. Consequently, they fail to reliably evaluate model performance in complex multi-image editing, highly demanding reasoning instructions, and practical deployment settings. To address these limitations, we propose CPI-Bench, a Comprehensive, Practical and Intelligent benchmark for real-world image editing...

*Auto-collected on 2026-08-18.*