MotiMotion: Motion-Controlled Video Generation with Visual Reasoning

Forum topic · 小凯 · 2026-05-23

Summary

MotiMotion is a new framework that reformulates motion-controlled image-to-video generation as a "reason first, generate later" problem. Existing motion-control I2V models strictly follow user-provided trajectories, which are often sparse, imprecise, and causally incomplete, leading to unnatural or illogical results, especially missing secondary causal consequences. To address this, MotiMotion leverages a training-free vision-language reasoner to refine primary trajectories in image space coordinates and to imagine plausible secondary motions, encouraging causally grounded and commonsense-consistent interactions. The framework also introduces a confidence-aware control scheme that adjusts guidance strength: tight adherence to the plan when confidence is high, and reliance on internal generative priors to fix artifacts when confidence is low. The authors further contribute MotiBench, an interaction-centric I2V benchmark composed of scenes where new events are triggered by motion. VLM-based evaluation and human studies on MotiBench show that MotiMotion produces videos with more plausible object behaviors and interactions than prior methods.

Paper Overview

Research Area: Computer Vision (CV)
Authors: Hsin-Ying Lee, Hanwen Jiang, Yiqun Mei
Release Date: 2025-05-23
arXiv: 2505.17386

Abstract

Current motion-controlled image-to-video generation models strictly follow user-provided trajectories, but these trajectories are often sparse, imprecise, and causally incomplete. This dependency frequently produces unnatural or illogical results, particularly omitting secondary causal consequences. To address this, the authors introduce MotiMotion, a new framework that reformulates motion control as a "reason first, generate later" problem.

To encourage causally grounded and commonsense-consistent interactions, the framework leverages a training-free vision-language reasoner (VLM) to:

1. Refine the image-space coordinates of primary trajectories. 2. "Imagine" plausible secondary motions that should logically follow.

To further improve motion naturalness, MotiMotion proposes a confidence-aware control scheme that modulates guidance strength:

When confidence is high, the model tightly follows the plan.
When confidence is low, the model falls back on its internal generative priors to correct artifacts.

To support systematic evaluation, the authors curate a new image-to-video benchmark called MotiBench, composed of interaction-centric scenes in which new events are triggered by motion.

Evaluation

Both VLM-based evaluations and human studies conducted on MotiBench demonstrate that MotiMotion produces videos with more plausible object behaviors and interactions, outperforming existing approaches.