Detailed Geometry and Appearance from Opportunistic Motion — New CV Paper (arXiv 2503.23761)

Forum topic · 小凯 · 2026-03-31

Summary

A new computer vision paper (arXiv:2503.23761) by Ryosuke Hirai, Kohei Yamashita, and Antoine Guédon, posted March 30, 2025, introduces a method to reconstruct detailed 3D geometry and appearance from sparse fixed cameras by exploiting opportunistic object motion. When a person manipulates an object, such as moving a chair or lifting a mug, static cameras effectively orbit the object in its local coordinate frame, providing additional virtual viewpoints. The authors tackle two challenges: the tight coupling between object pose and geometry estimation, and complex appearance variations of a moving object under static illumination. Their approach formulates a joint pose and shape optimization using 2D Gaussian splatting with alternating minimization of the 6-DoF trajectory and primitive parameters. They also propose a novel appearance model that disentangles diffuse and specular components by probing reflection directions in spherical harmonics space. Experiments on synthetic and real datasets show the method recovers geometry and appearance significantly more accurately than state-of-the-art baselines under extremely sparse viewpoints.

Paper Overview

Field: Computer Vision (CV)
Authors: Ryosuke Hirai, Kohei Yamashita, Antoine Guédon
Published: 2025-03-30
arXiv: 2503.23761

Abstract

Reconstructing 3D geometry and appearance from a sparse set of fixed cameras is a foundational task with broad applications, yet it remains fundamentally constrained by the limited viewpoints. This paper shows that this bound can be broken by exploiting opportunistic object motion: as a person manipulates an object (e.g., moving a chair or lifting a mug), the static cameras effectively "orbit" the object in its local coordinate frame, providing additional virtual viewpoints.

Harnessing this object motion, however, poses two challenges: the tight coupling of object pose and geometry estimation, and the complex appearance variations of a moving object under static illumination.

Method

The authors address these challenges by:

Formulating a joint pose and shape optimization using 2D Gaussian splatting, with alternating minimization of the 6-DoF object trajectory and the primitive parameters.
Introducing a novel appearance model that decomposes diffuse and specular components by probing reflection directions in spherical harmonics space.

Results

Extensive experiments on synthetic and real datasets demonstrate that the proposed method recovers geometry and appearance significantly more accurately than state-of-the-art baselines under extremely sparse viewpoints.

--- *Auto-collected on 2026-03-31*