OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

Forum topic · 小凯 · 2026-09-22

Summary

OmniVBench is a new benchmark and Omni-R2V dataset for evaluating and training omni reference-to-video (R2V) generation models. The paper addresses gaps in existing benchmarks, which cover limited reference types and compositions and only assess holistic reference consistency. OmniVBench expands evaluation to 7 task families and 18 fine-grained tasks spanning content, motion, style, structure, narrative, and multi-reference settings, and introduces element-anchored evaluation with 12,172 case-specific checks that verify whether reference elements are faithfully preserved, correctly disentangled and routed to targets, and properly realized per instructions. The accompanying Omni-R2V dataset, largely derived from a large-scale professional video library, contains 340,000 processed training samples, plus a scalable pipeline for constructing reference-target pairs. Large-scale evaluation of leading open-source and closed-source R2V models reveals significant performance gaps across task families and evaluation dimensions, highlighting current R2V limitations. Paper: arXiv 2609.22069.

Paper Overview

Field: Computer Vision Authors: Wenxue Li, Peiyan Guan, Haoyang Jiang, Junxian Cai, Hualuo Liu, Chunjie Zhang, Chong Guan, Kai Huang, Songlian Li, Taiyi Wu, Yongjian Yu, Xiaotong Zhao, Alan Zhao, Eric Liu, Xi Chen, Yu Liu, Lei Zhu Published: 2026-09-18 arXiv: 2609.22069

Abstract

Reference-to-video (R2V) generation is evolving toward increasingly general and versatile reference control, giving rise to the emerging paradigm of omni R2V generation. However, existing benchmarks fall short of these emerging capabilities: their test cases cover limited reference types and compositions, and their evaluation protocols largely assess holistic reference consistency, overlooking whether reference factors are properly preserved, disentangled, and routed. Meanwhile, the high cost of constructing omni R2V training data makes suitable training resources scarce.

To address these gaps, the authors introduce OmniVBench and the Omni-R2V Dataset for evaluating and training omni R2V models.

Key Contributions

OmniVBench expands R2V evaluation across broader reference types, fine-grained control tasks, and richer reference combinations, covering 7 task families and 18 fine-grained tasks spanning content, motion, style, structure, narrative, and multi-reference settings.
Element-anchored evaluation: 12,172 case-specific checkpoints that assess whether the expected reference elements are (1) faithfully preserved, (2) correctly disentangled and bound to their targets, and (3) properly realized according to instructions.
Omni-R2V Dataset: primarily derived from a large-scale professional video footage library, containing 340,000 processed training samples, providing an industrial-grade training resource for diverse R2V tasks.
Task-specific reference-target pair construction pipeline, offering a practical and scalable path for building omni R2V training data.

Findings

Large-scale evaluation of state-of-the-art open-source and closed-source R2V models reveals clear performance gaps across OmniVBench's task families and evaluation dimensions, underscoring the limitations of current R2V models.