AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation

Forum topic · 小凯 · 2026-04-12

Summary

AVGen-Bench (arXiv:2504.07857) is a task-driven benchmark for Text-to-Audio-Video (T2AV) generation, introduced by Ziwei Zhou, Zeyuan Lai, and Rui Wang in April 2025. Existing evaluation methods assess audio and video separately or rely on coarse embedding similarity, missing the fine-grained joint correctness required by realistic prompts. AVGen-Bench provides high-quality prompts spanning 11 real-world categories and pairs them with a multi-granular evaluation framework combining lightweight specialist models with Multimodal Large Language Models (MLLMs). The framework measures everything from perceptual quality to fine-grained semantic controllability. Evaluation results reveal a notable gap between strong audio-visual aesthetics and weak semantic reliability, with persistent failures in text rendering, speech coherence, physical reasoning, and a widespread inability to control musical pitch. The benchmark aims to standardize and deepen T2AV evaluation for the research community.

Paper Overview

Field: NLP / Multimodal Generation Authors: Ziwei Zhou, Zeyuan Lai, Rui Wang Published: 2025-04-10 arXiv: 2504.07857

Abstract

Text-to-Audio-Video (T2AV) generation is rapidly becoming a core interface for media creation, yet its evaluation remains fragmented. Existing benchmarks largely assess audio and video in isolation or rely on coarse embedding similarity, failing to capture the fine-grained joint correctness required by realistic prompts.

The authors introduce AVGen-Bench, a task-driven benchmark for T2AV generation featuring high-quality prompts across 11 real-world categories. To support comprehensive assessment, they propose a multi-granular evaluation framework that combines lightweight specialist models with Multimodal Large Language Models (MLLMs), enabling evaluation from perceptual quality to fine-grained semantic controllability.

Key Findings

The evaluation reveals a pronounced gap between strong audio-visual aesthetics and weak semantic reliability, including:

Persistent failures in text rendering
Breakdowns in speech coherence
Weaknesses in physical reasoning
A widespread inability to follow musical pitch control instructions