FlowHMR: Physically Plausible Motion Capture from Video (arXiv 2610.03691)

Forum topic · 小凯 · 2026-10-06

Summary

FlowHMR is a new framework for recovering physically plausible global 3D human motion from monocular video, presented in arXiv paper 2610.03691. Instead of directly regressing motion from video — which suffers from depth ambiguity and collapses toward averaged solutions — FlowHMR reframes video motion capture as video-conditioned motion generation. The authors first pretrain a flow matching model that generates diverse motion candidates from an input video, then apply post-training with Group Relative Policy Optimization (GRPO) using two rewards: a faithfulness reward encouraging consistency with the video, and a tracking reward favoring motions that a physics-based controller can successfully track. The team also introduces Wild-4K, a dataset of roughly 4,000 diverse web videos for evaluating in-the-wild human motion recovery. Experiments show FlowHMR outperforms state-of-the-art methods in motion faithfulness and achieves an 82.47% physics tracking success rate versus 62.82% for the strongest baseline, GVHMR.

Paper Overview

Field: Computer Vision Authors: Zhanke Wang, Chengfeng Zhao, Qing Shuai, Jingzhong Lin, Heng Li, Zeyu Ling, Yuxin Wen, Jing Li, Di Kang, Chunchao Guo, Linchao Bao Released: 2026-10-02 arXiv: 2610.03691

Summary

The authors present FlowHMR, a framework for recovering physically plausible global 3D human motion from monocular video. Previous learning-based methods typically regress human motion directly from video and train the network with geometric supervision. However, recovering human motion from monocular video is inherently ambiguous in depth, and direct regression tends to collapse toward an averaged solution. Moreover, the recovered motions are not guaranteed to be physically plausible, so physics-based tracking of them often fails.

To address these challenges, FlowHMR formulates video motion capture as a video-conditioned motion generation problem:

1. Pretraining: A flow matching model is pretrained to generate motion conditioned on video. 2. Candidate generation: Given an input video, the pretrained model generates diverse motion candidates — though not all candidates are faithful to the video or physically trackable. 3. Post-training with GRPO: The model is refined using Group Relative Policy Optimization with two rewards:

A faithfulness reward encouraging consistency with the input video.
A tracking reward preferring motions that can be successfully tracked by a physics-based controller.

Together, these rewards shift the model's output preference so that the post-trained model produces motions that are both faithful to the input video and physically plausible.

Wild-4K Dataset

The paper also introduces Wild-4K, a large-scale, diverse dataset of approximately 4,000 web videos for evaluating in-the-wild human motion recovery.

Results

Qualitative and quantitative experiments on Wild-4K show that FlowHMR outperforms the state of the art in overall motion faithfulness, with a physics tracking success rate of 82.47%, compared to 62.82% for the strongest baseline, GVHMR.

---

*Auto-collected on 2026-10-06.*

FlowHMR: Physically Plausible Motion Capture from Video (arXiv 2610.03691)

Summary

Paper Overview

Summary

Wild-4K Dataset

Results

Tags