Paper Overview
Field: Computer Vision Authors: Leyi Wu, Pengjun Fang, Kai Sun Release date: 2025-04-17 arXiv: 2504.13082
Abstract
Video generation has advanced rapidly, with recent methods producing increasingly convincing animated results. However, existing benchmarks—largely designed for realistic videos—struggle to evaluate animation-style generation with its stylized appearance, exaggerated motion, and character-centric consistency. Moreover, they also rely on fixed prompt sets and rigid pipelines, offering limited flexibility for open-domain content and custom evaluation needs.
To address this gap, the authors introduce AnimationBench, the first systematic benchmark for evaluating animation image-to-video generation. AnimationBench operationalizes the Twelve Basic Principles of Animation and IP Preservation into measurable evaluation dimensions, together with Broader Quality Dimensions including semantic consistency, motion plausibility, and camera motion consistency.
The benchmark supports both standardized closed-set evaluation for reproducible comparisons and flexible open-set evaluation for diagnostic analysis, leveraging vision-language models for scalable assessment. Extensive experiments demonstrate that AnimationBench aligns closely with human judgments and reveals animation-specific quality differences that realism-oriented benchmarks overlook, enabling more informative and discriminative evaluation of state-of-the-art I2V models.
Key Contributions
- First systematic benchmark targeting animation-style I2V generation
- Formalizes the Twelve Basic Principles of Animation and IP Preservation as measurable dimensions
- Dual evaluation modes: closed-set (reproducible) and open-set (custom diagnostics)
- Scalable assessment via vision-language models, with strong alignment to human judgment