InstructMesh: Interactive Refinement of Generative 3D Models for Fabrication

Forum topic · 小凯 · 2026-09-01

Summary

InstructMesh is an interactive post-generation refinement tool that enables selective repair of generative 3D models for fabrication. While modern generative AI can create 3D models from text or images, these outputs prioritize visual plausibility over geometric accuracy, often producing flaws that compromise their intended physical use. InstructMesh lets users select regions and apply targeted operations—such as opening or sealing voids or adjusting local thickness—invoked through natural language prompts or slider controls. By operating directly on an intermediate latent representation, the system enables robust geometric corrections without expert modeling skills. The authors first analyzed common fabrication-related failure modes in state-of-the-art generative tools to inform the design, then conducted two user studies showing that novices can perform fabrication-oriented repairs on generative outputs, with a preference for a hybrid interface combining sliders and natural language input.

InstructMesh: Selective Refinement of Generative 3D Models for Fabrication

Field: Machine Learning

Authors: Faraz Faruqi, Ahmed Katary, Demircan Tas, Theresa Hradilak, Ning Zhang, Jiaji Li, Fabian Manhardt, Martin Nisser, Vrushank Phadnis, Ruofei Du, Federico Tombari, Megan Hofmann, Stefanie Mueller

Published: 2026-08-28

arXiv: 2608.28534

Abstract

Recent advances in generative AI allow users to create 3D models from text or images. However, these models prioritize visual plausibility over geometric accuracy, often generating results with flaws that compromise their intended use post-fabrication. We present InstructMesh, an interactive post-generation refinement tool that enables selective repair of generative 3D models through region selection and targeted operations, such as opening or sealing voids, or adjusting local thickness. Users can invoke edit operations via natural language prompts or slider controls. By operating directly on the intermediate latent representation, InstructMesh allows users to apply robust geometric corrections without requiring expert modeling skills. To inform our design, we first analyze common fabrication-related failure modes in the outputs of state-of-the-art generative tools. We then conduct two user studies demonstrating that novices can use InstructMesh to perform fabrication-oriented repairs on generative outputs, and reveal a preference for hybrid interfaces combining slider controls and natural language input.

---

*Auto-collected on 2026-09-01*

InstructMesh: Interactive Refinement of Generative 3D Models for Fabrication

Summary

InstructMesh: Selective Refinement of Generative 3D Models for Fabrication

Abstract

Tags