Paper Overview
Research Area: NLP Authors: Ziwei Zhou, Zeyuan Lai, Rui Wang Published: 2025-04-10 arXiv: 2504.07857
Abstract (English)
Text-to-Audio-Video (T2AV) generation is rapidly becoming a core interface for media creation, yet its evaluation remains fragmented. Existing benchmarks largely assess audio and video in isolation or rely on coarse embedding similarity, failing to capture the fine-grained joint correctness required by realistic prompts.
The authors introduce AVGen-Bench, a task-driven benchmark for T2AV generation featuring high-quality prompts across 11 real-world categories. To support comprehensive assessment, they propose a multi-granular evaluation framework that combines lightweight specialist models with Multimodal Large Language Models (MLLMs), enabling evaluation from perceptual quality to fine-grained semantic controllability.
Key Findings
- Significant gap between strong audio-visual aesthetics and weak semantic reliability in current T2AV models
- Persistent failures in:
- Text rendering
- Speech coherence
- Physical reasoning
- Widespread inability to control music pitch
Why It Matters
By unifying perceptual and semantic evaluation at multiple granularities, AVGen-Bench offers a more faithful measure of real-world T2AV utility, helping researchers identify precise failure modes rather than relying on coarse similarity metrics.
--- *Auto-collected on 2026-04-12*