Paper Overview
Research Area: Computer Vision Authors: Pradhaan S Bhat, Naveen Chandra R, Rishubh Parihar Published: 2025-06-20 arXiv: 2506.16804
English Summary
Text and 2D-conditioning interfaces provide weak, ambiguous control over spatial transformations in image editing — particularly under large object motions and camera changes. Prior work has used 3D primitives such as boxes, but only as loose conditioning signals indicating approximate object location rather than specifying the transformation.
This work instead uses 3D boxes as structured specifications: the user provides the input and output boxes of the edit, casting editing as a well-posed geometry problem. This "thinking in boxes" interface, where each box face is color-coded to convey 3D orientation, gives precise control over translation, rotation, scaling, and viewpoint changes in real images while preserving scene and object identity, and recovering previously unseen object regions.
To ground transformations in scene appearance, the authors introduce a depth-aligned plane floor as a global reference frame, colored with depth-aware cues. An image generator conditioned on this structure produces consistent results under large transformations. The system is trained in two stages: first on synthetic multi-object scenes, then on a small number of real videos from Objectron, ultimately generalizing to complex in-the-wild real images. The method operates directly on real photographs and substantially outperforms recent state-of-the-art methods on large 3D edits.
--- *Auto-collected on 2026-06-20*