HOMIE: Human-Object Centric Video Personalization via Multimodal Integration

Forum topic · 小凯 · 2026-07-22

Summary

HOMIE is a framework for human-object centric video personalization (HOCVP), a core task in subject-driven video generation. The paper addresses two key limitations of existing methods: first, inter-subject personalization approaches struggle to balance high subject fidelity with accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts like logos. Second, most prior work lacks mechanisms to understand intra-subject references such as OCR maps and multi-view inputs that could enhance subject fidelity. HOMIE handles both inter- and intra-subject input settings in a unified manner, proposing an improved MLLM integration strategy to extract reference-level relational knowledge without compromising the text encoder's controllability or requiring costly realignment. The framework also introduces global multimodal guidance and modality-reference embedding, achieving state-of-the-art performance. Authored by Yiyang Cai, Nan Chen, Rongchang Xie, Junwen Pan, Chunyang Jiang and 6 others, the paper is available on arXiv under cs.CV (arXiv:2607.18217).

Paper Overview

Research Area: Computer Vision (CV) Authors: Yiyang Cai, Nan Chen, Rongchang Xie, Junwen Pan, Chunyang Jiang, et al. (11 authors total) Published: 2026-07-20 arXiv: 2607.18217 Category: cs.CV

Introduction

Human-object centric video personalization (HOCVP) is a core task in subject-driven video generation. However, existing methods suffer from two key limitations:

1. Inter-subject personalization challenges: Most methods struggle to balance high subject fidelity with accurate interaction patterns between humans and diverse objects — especially when objects represent abstract concepts such as logos.

2. Underused intra-subject references: While intra-subject references (e.g., OCR maps, multi-view inputs) show promise for enhancing subject fidelity, most existing work lacks mechanisms to understand this latent correspondence.

The HOMIE Approach

To address both challenges, the authors propose HOMIE, a framework that handles inter-subject and intra-subject input settings in a unified manner. Compared to prior methods, HOMIE:

Proposes a better MLLM integration strategy to extract reference-level relational knowledge, without compromising the text encoder's controllability or producing costly re-alignment.
Introduces global multimodal guidance.
Employs modality-reference embedding.

Together, these components achieve state-of-the-art performance on HOCVP.