Summary
A cs.CV paper (arXiv 2506.17584) by Pradhaan S Bhat, Naveen Chandra R, and Rishubh Parihar proposes Thinking in Boxes, a 3D image editing method for real photographs. Instead of treating 3D primitives as loose conditioning signals, the interface uses pairs of 3D boxes—an input box and an output box—turning editing into a well-defined geometric problem. Each box face is color-coded to encode orientation, giving users precise control over translation, rotation, scaling, and viewpoint changes while preserving object and scene identity and revealing previously unseen regions. A depth-aligned planar floor, shaded with depth-aware hints, serves as a global reference frame to anchor transforms to scene appearance. The model is trained in two stages: pretraining on synthetic multi-object scenes, then finetuning on a small set of real-world Objectron videos. The method operates directly on real photos and significantly outperforms recent state-of-the-art approaches on large-magnitude 3D edits.
Overview
- Field: cs.CV
- Authors: Pradhaan S Bhat, Naveen Chandra R, Rishubh Parihar
- arXiv: 2506.17584
Key Ideas
Text- and 2D-conditioned interfaces offer weak and ambiguous control over spatial transformations in image editing, especially for large object motions or camera viewpoint changes. Prior work introduced 3D primitives such as cuboids, but only as loose conditioning signals that roughly indicate object placement rather than precisely specifying the transformation.This paper takes the opposite approach: it uses 3D boxes as a structured canonical. The user provides an input box and an output box (before/after), converting the editing process into a well-defined geometric problem. In this "Thinking in Boxes" interface, each face of a box is color-coded to encode 3D orientation, giving users precise control over translation, rotation, scaling, and viewpoint changes in real images, while preserving scene and object identity and recovering previously unseen object regions.
Method
- A depth-aligned planar floor acts as a global reference frame, shaded with depth-aware hints, anchoring transformations to scene appearance.
- Guided by this structure, the image generator produces highly consistent results even under large transformations.
- Two-stage training: pretraining on synthetic multi-object scenes, followed by finetuning on a small set of real-world videos from the Objectron dataset, enabling generalization to complex, in-the-wild real photographs.
Results
The method works directly on real photos and significantly outperforms recent state-of-the-art methods on large-magnitude 3D editing tasks.
Plain-Language Explanation
Imagine you want to move a table in a photo. Older methods are like pointing directions through frosted glass—"maybe around here." This approach gives you two transparent 3D boxes: one marking "now," one marking "where it should go." Each face of the box is painted a different color so you know front/back/left/right/up/down, as clear as building with blocks. A floor plane provides depth like invisible coordinate paper across the room, so the AI understands exactly how to "move house"—and the result looks realistic, without glitches.
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178207990