Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models (SEIG)

Forum topic · 小凯 · 2026-06-03

Summary

This paper investigates whether pretrained vision-language models (VLMs) can perform executable inverse graphics from a single image by reconstructing a scene as an editable Blender program, without specialized 2D/3D foundation models, differentiable rendering, or multi-view supervision. The authors propose Staged Executable Inverse Graphics (SEIG), an agentic framework that progressively refines scene factors—geometry, materials, composition, and lighting—directly in executable Blender code space. Inverse graphics is a longstanding, highly underconstrained problem aiming to reconstruct images as editable 3D scenes that can be rendered, relit, and manipulated. The framework was evaluated across diverse scenes using reconstruction metrics covering pixel-level, perceptual, and semantic fidelity. Experiments show that staged reconstruction significantly improves reconstruction fidelity, highlighting the importance of task decomposition for general-purpose VLMs performing executable inverse graphics. The paper also demonstrates the reconstructed editable Blender scenes in various downstream applications. Authors: Guangzhao He, Rundong Luo, Wei-Chiu Ma. arXiv: 2506.00001.

Paper Overview

Field: Computer Vision (CV) Authors: Guangzhao He, Rundong Luo, Wei-Chiu Ma Published: 2026-06-03 arXiv: 2506.00001

Abstract (original)

Inverse graphics is a longstanding and highly underconstrained problem that seeks to reconstruct images as editable 3D scenes which can be rendered, relit, and manipulated. In this work, we investigate whether pretrained vision-language models (VLMs) can perform executable inverse graphics directly from a single image by reconstructing a scene as an editable Blender program, without relying on specialized 2D or 3D foundation models, differentiable rendering, or multi-view supervision.

We introduce Staged Executable Inverse Graphics (SEIG), an agentic framework that reconstructs a 3D scene from a single image by progressively refining scene factors including geometry, materials, composition, and lighting directly in executable Blender code space.

We evaluate our framework across diverse scenes using reconstruction metrics covering pixel-level, perceptual, and semantic fidelity.

Key Findings

Staged reconstruction significantly improves reconstruction fidelity, demonstrating that task decomposition is important for general-purpose VLMs to perform executable inverse graphics.
The reconstructed editable Blender scenes support a range of downstream applications, including rendering, relighting, and scene manipulation.
The approach requires no specialized 2D/3D foundation models, differentiable rendering, or multi-view supervision.

Chinese Summary (from forum post)

逆向图形学是一个长期存在且高度欠约束的问题，旨在将图像重建为可编辑的3D场景。本文提出分阶段可执行逆向图形学（SEIG），一个通过在可执行的Blender代码空间中逐步优化几何、材质、构图和光照等场景因素，从单张图像重建3D场景的智能体框架。实验表明，分阶段重建显著提升了重建保真度，凸显了任务分解对于通用VLM执行可执行逆向图形学的重要性。

---

*Auto-collected on 2026-06-03*