Summary
AVGen-Bench (arXiv 2504.07073, Ziwei Zhou, Zeyuan Lai, Rui Wang) is a task-driven benchmark for evaluating Text-to-Audio-Video (T2AV) generation. Existing benchmarks mostly assess audio and video separately or rely on coarse embedding similarity, missing the fine-grained joint correctness that realistic prompts require. AVGen-Bench provides high-quality prompts spanning 11 real-world categories and introduces a multi-granular evaluation framework that combines lightweight specialist models with Multimodal Large Language Models (MLLMs), measuring everything from perceptual quality to fine-grained semantic controllability. Evaluation results reveal a pronounced gap between strong audio-visual aesthetics and weak semantic reliability: models persistently fail at text rendering, speech coherence, and physical reasoning, and universally break down on musical pitch control. The benchmark offers a standardized, comprehensive methodology for T2AV model assessment and highlights key weaknesses for future research in generative audio-visual media creation.
Paper Overview
Field: AI
Authors: Ziwei Zhou, Zeyuan Lai, Rui Wang
Published: 2025-04-10
arXiv:
2504.07073Abstract
Text-to-Audio-Video (T2AV) generation is rapidly becoming a core interface for media creation, yet its evaluation remains fragmented. Existing benchmarks largely assess audio and video in isolation or rely on coarse embedding similarity, failing to capture the fine-grained joint correctness required by realistic prompts.
The authors introduce AVGen-Bench, a task-driven benchmark for T2AV generation featuring high-quality prompts across 11 real-world categories. To support comprehensive assessment, they propose a multi-granular evaluation framework that combines lightweight specialist models with Multimodal Large Language Models (MLLMs), enabling evaluation from perceptual quality to fine-grained semantic controllability.
Key Findings
The evaluation reveals a pronounced gap between strong audio-visual aesthetics and weak semantic reliability, including:
- Persistent failures in text rendering
- Breakdowns in speech coherence
- Weaknesses in physical reasoning
- A universal collapse in musical pitch control
Links
- arXiv: https://arxiv.org/abs/2504.07073
---
*Auto-collected on 2025-04-11*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177169739