MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation

Forum topic · 小凯 · 2026-06-26

Summary

MVTrack4Gen is a motion-aware training framework for novel-view video generation introduced by Joung Bin Lee, Jaewoo Jung, and Jongmin Lee (arXiv 2606.19227). The method synthesizes a new-view video from a monocular reference video along a target camera trajectory, addressing the trade-off between explicit 3D representation methods (limited by inaccurate reconstruction of dynamic objects) and camera-conditioning-only methods (high visual quality but weak geometric and motion consistency). The key insight is that specific attention layers encode strong correspondence cues: query features attend to key features at geometrically corresponding positions across views and time, and misalignment of these correspondences causes motion inconsistency. MVTrack4Gen routes these attention features to an auxiliary multi-view tracking head and jointly trains the camera-conditioned video diffusion model with a point tracking objective. This explicit motion-aware supervision improves adherence to reference-video motion and cross-view geometric consistency, achieving state-of-the-art geometric consistency and competitive camera accuracy across diverse benchmarks.

Overview

Research area: Computer Vision Authors: JoungBin Lee, Jaewoo Jung, Jongmin Lee Published: 2026-06-25 arXiv: 2606.19227

Problem

Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video.

Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos.
Camera-conditioning-only methods can achieve high visual quality but often struggle to preserve geometric and motion consistency.

Method

MVTrack4Gen (Multi-View point Tracking for Novel-View Generation) is a motion-aware training framework that leverages multi-view point tracking as an additional geometric and motion supervision signal for camera-conditioned novel-view video diffusion models.

Key findings and design:

Specific attention layers encode strong correspondence cues, where query features attend to key features at geometrically corresponding positions across views and across time. Misalignment of these correspondences leads to motion inconsistency.
Based on this observation, the framework routes these attention features to an auxiliary multi-view tracking head and jointly trains the diffusion model with a point tracking objective.
By explicitly enhancing these motion-aware correspondences, MVTrack4Gen improves existing models so they better follow the motion in the reference view and maintain cross-view geometric consistency.

Results

On diverse benchmarks, MVTrack4Gen achieves state-of-the-art geometric consistency and competitive camera accuracy.

---

*Originally posted on zhichai.net, auto-collected 2026-06-26.*