Overview
- Field: Computer Vision
- Authors: Pang-Chi Lo, Jhao-Syun Lai, Tsung-Wei Ke
- Published: 2026-11-05
- arXiv: 2611.08436
- Motion estimation: Evaluated on three dynamic-scene datasets—Kubric, Waymo, and Point Odyssey—SM4RT achieves the best performance on most metrics compared with prior baselines.
- Geometry reconstruction: Benchmarked on five dynamic scene datasets, SM4RT achieves competitive performance with state-of-the-art methods.
Abstract
We introduce SM4RT, a multi-task cascade feed-forward model for dynamic scene reconstruction from RGB videos. While existing feed-forward approaches typically address geometry reconstruction and motion estimation separately—and often predict motions in a single coordinate frame that must account for camera movement—SM4RT solves these tasks sequentially by first reconstructing 3D geometry and then bootstrapping motion estimation from the predicted structure.By expressing each pixel's 3D motion in its corresponding camera frame, SM4RT reduces ambiguity arising from camera motion. To address the sparsity of real-world motion labels, SM4RT further incorporates a weakly supervised loss that regularizes unlabeled point motions using labeled point positions at the other time step.
Key Results
Takeaway
SM4RT's unified framework demonstrates that combining geometry reconstruction and motion estimation in a cascade not only avoids performance trade-offs but also enhances motion estimation through improved geometric reasoning.--- *Auto-collected on 2026-11-08.*