MotiMotion: Reasoning-Guided Motion Control for Image-to-Video Generation

Forum topic · 小凯 · 2026-05-25

Summary

MotiMotion is a new framework for motion-controlled image-to-video generation that reformulates motion control as a reason-first, generate-second process. Instead of rigidly following sparse, imprecise user paths, the method uses a training-free visual-language reasoner to refine primary path coordinates and infer plausible secondary causal motions, encouraging common-sense interactions. A confidence-aware control scheme modulates guidance strength: high-confidence plans are followed closely, while low-confidence inputs leverage the model's internal generative priors to fix artifacts. The authors also introduce MotiBench, an interaction-centric image-to-video benchmark where new events are triggered by motion. Both VLM-based evaluation and human studies on MotiBench show that MotiMotion produces videos with more plausible object behavior and interactions, outperforming existing methods. Paper: arXiv 2505.14486.

Paper Overview

Field: Computer Vision (CV) Authors: Lee Hsin-Ying, Hanwen Jiang, Yiqun Mei Published: 2026-05-25 arXiv: 2505.14486

Abstract (English translation)

Current motion-controlled image-to-video generation models rigidly follow user-provided paths, which are often sparse, imprecise, and causally incomplete. This dependence often yields unnatural or implausible results, particularly missing secondary causal consequences. To address this, we introduce MotiMotion, a new framework that reformulates motion control as reason-first, generate-second.

To encourage causal, common-sense-grounded interactions, we leverage a training-free visual-language reasoner to refine the image-space coordinates of primary paths and to infer plausible secondary motions. To further improve motion naturalness, we propose a confidence-aware control scheme that modulates guidance strength, enabling the model to closely follow high-confidence plans while using its internal generative priors to correct artifacts under low-confidence inputs.

To support systematic evaluation, we curate a new image-to-video benchmark, MotiBench, consisting of interaction-centric scenes in which new events are triggered by motion. Both VLM-based evaluation and human studies on MotiBench demonstrate that MotiMotion produces videos with more plausible object behavior and interactions, outperforming existing methods.

Key Contributions

Reason-first motion control: reframes motion control as reasoning before generation rather than blindly following input paths
Training-free visual-language reasoner: refines primary path coordinates and hallucinates plausible secondary causal motions
Confidence-aware guidance: balances adherence to high-confidence plans with self-correction via generative priors
MotiBench: a new interaction-centric benchmark where motions trigger new events