OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation

Forum topic · 小凯 · 2026-04-15

Summary

OmniShow is an end-to-end framework for Human-Object Interaction Video Generation (HOIVG), synthesizing high-quality videos of humans interacting with objects conditioned on text, reference images, audio, and pose. The task has practical value for automated content creation in e-commerce demonstrations, short-video production, and interactive entertainment. The framework introduces unified channel conditioning for efficient injection of images and poses, and gated local-context attention to achieve precise audio-visual synchronization. To address data scarcity, the authors develop a disentangle-then-joint training strategy. The work also contributes HOIVG-Bench, a dedicated benchmark for evaluating human-object interaction video generation. Authored by Donghao Zhou, Guisheng Liu, Hao Yang, and colleagues including researchers from CUHK-affiliated teams (Chi-Wing Fu, Pheng-Ann Heng), the paper is available on arXiv (2604.11804) in the computer vision category (cs.CV).

[Paper] OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation

Paper Overview

Research Area: cs.CV Authors: Donghao Zhou, Guisheng Liu, Hao Yang, Jiatong Li, Jingyu Lin, Xiaohu Huang, Yichen Liu, Xin Gao, Cunjian Chen, Shilei Wen, Chi-Wing Fu, Pheng-Ann Heng Published: 2026-04-13 arXiv: 2604.11804

Summary

This work studies Human-Object Interaction Video Generation (HOIVG), which aims to synthesize high-quality human-object interaction videos conditioned on text, reference images, audio, and pose. The task holds significant practical value for automating content creation in real-world applications such as e-commerce demonstrations, short video production, and interactive entertainment.

The authors propose OmniShow, an end-to-end framework that coordinates multimodal conditions and delivers industrial-level performance. Key technical contributions include:

Unified channel conditioning: enables efficient injection of reference images and pose information into the generation model.
Gated local-context attention: ensures precise audio-visual synchronization.
Disentangle-then-joint training strategy: developed to cope with data scarcity in this domain.
HOIVG-Bench: a dedicated benchmark established for evaluating human-object interaction video generation.

Original Abstract (excerpt)

> In this work, we study Human-Object Interaction Video Generation (HOIVG), which aims to synthesize high-quality human-object interaction videos conditioned on text, reference images, audio, and pose. This task holds significant practical value for automating content creation in real-world applications, such as e-commerce demonstrations, short video production, and interactive entertainment.

--- *Auto-collected on 2026-04-15*

OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation

Summary

Paper Overview

Summary

Original Abstract (excerpt)

Tags