Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models

Forum topic · 小凯 · 2026-06-03

Summary

This paper investigates whether pretrained vision-language models (VLMs) can perform executable inverse graphics directly from a single image, reconstructing a scene as an editable Blender program without specialized 2D/3D foundation models, differentiable rendering, or multi-view supervision. The authors introduce Staged Executable Inverse Graphics (SEIG), an agentic framework that progressively refines scene factors—geometry, materials, composition, and lighting—directly in executable Blender code space. Inverse graphics is a longstanding, highly underconstrained problem aiming to recover editable 3D scenes that can be rendered, relit, and manipulated. The framework is evaluated across diverse scenes using reconstruction metrics spanning pixel-level, perceptual, and semantic fidelity. Experiments show that staged reconstruction significantly improves reconstruction fidelity, highlighting the importance of task decomposition for enabling general-purpose VLMs to perform executable inverse graphics. The authors also demonstrate the reconstructed editable Blender scenes in various downstream applications. Paper by Guangzhao He, Rundong Luo, and Wei-Chiu Ma, available on arXiv as 2506.00001.

Paper Overview

Field: Computer Vision Authors: Guangzhao He, Rundong Luo, Wei-Chiu Ma Published: 2026-06-03 arXiv: 2506.00001

Original Abstract

Inverse graphics is a longstanding and highly underconstrained problem that seeks to reconstruct images as editable 3D scenes which can be rendered, relit, and manipulated. In this work, we investigate whether pretrained vision-language models (VLMs) can perform executable inverse graphics directly from a single image by reconstructing a scene as an editable Blender program, without relying on specialized 2D or 3D foundation models, differentiable rendering, or multi-view supervision.

We introduce Staged Executable Inverse Graphics (SEIG), an agentic framework that reconstructs a 3D scene from a single image by progressively refining scene factors including geometry, materials, composition, and lighting directly in executable Blender code space.

Key Points

Reconstructs a single image into an editable Blender program rather than a static 3D representation
Requires no specialized 2D/3D foundation models, differentiable rendering, or multi-view supervision — only a pretrained VLM
Staged refinement: scene factors (geometry, materials, composition, lighting) are optimized progressively in executable code space
Evaluated across diverse scenes with pixel-level, perceptual, and semantic fidelity metrics
Staged reconstruction significantly improves fidelity, showing that task decomposition matters for general-purpose VLMs
Reconstructed editable Blender scenes support downstream applications such as relighting and manipulation

Chinese Summary (from source)

逆向图形学是一个长期存在且高度欠约束的问题，旨在将图像重建为可编辑的3D场景。本文提出的SEIG框架通过在可执行的Blender代码空间中逐步优化几何、材质、构图和光照等场景因素，从单张图像重建3D场景。实验表明，分阶段重建显著提升了重建保真度，凸显了任务分解对于通用VLM执行可执行逆向图形学的重要性。

---

*Auto-collected on 2026-06-03*