Paper Overview
Field: Computer Vision Authors: Pradhaan S Bhat, Naveen Chandra R, Rishubh Parihar Published: 2026-06-20 arXiv: 2506.16438
Abstract (translated)
Text- and 2D-conditioned interfaces for image editing offer weak, ambiguous control over spatial transformations, especially for large object movements and camera changes. Prior work has used 3D primitives (such as boxes), but only as loose conditioning signals indicating approximate object location rather than specifying the transformation. This paper instead uses 3D boxes as a structured specification: the user provides input and output boxes for the edit, turning editing into a well-defined geometric problem.
In this "Thinking in Boxes" interface, each box face is color-coded to convey 3D orientation, enabling precise control over translation, rotation, scaling, and viewpoint changes in real images while preserving scene and object identity and recovering previously unseen object regions.
To anchor transformations in scene appearance, the authors introduce a depth-aligned planar floor as a global reference frame, colored with depth-aware cues. Under this structured conditioning, an image generator produces consistent results under large transformations.
The system is trained in two stages—on synthetic multi-object scenes and a small set of Objectron real-world videos—and generalizes to complex in-the-wild real images. The method operates directly on real photographs and substantially outperforms recent state-of-the-art methods on large-scale 3D editing.
*Auto-collected on 2026-06-21.*