Articraft: An Agentic System for Scalable Articulated 3D Asset Generation

Forum topic · 小凯 · 2026-05-16

Summary

Articraft is an agentic system that uses large language models (LLMs) to generate articulated 3D assets at scale, addressing the scarcity of large, diverse datasets for articulated object understanding. The approach reduces articulated 3D asset generation to writing a program that builds the asset. The LLM writes code against a domain-specific SDK for defining parts, composing geometry, specifying joints, and writing tests that validate the resulting assets. A harness exposes a restricted workspace and interface to the LLM, validates generated assets, and returns structured feedback, so the LLM is not distracted by details like writing URDF files or managing complex software environments. Experiments show the resulting assets outperform state-of-the-art articulated asset generators and general coding agents. Using Articraft, the authors build Articraft-10K, a curated dataset of over 10K articulated assets across 245 categories, and demonstrate its utility for training articulated asset models and downstream applications such as robot simulation and virtual reality. Paper: arXiv 2505.08631.

Overview

Field: Computer Vision
Authors: Matt Zhou, Ruining Li, Xiaoyang Lyu
arXiv: 2505.08631

Abstract

A bottleneck in learning to understand articulated 3D objects is the lack of large and diverse datasets. This paper proposes leveraging large language models (LLMs) to close this gap and generate articulated assets at scale.

Approach

The problem of generating an articulated 3D asset is reduced to writing a program that builds it.
Articraft is a new agentic system that writes such programs automatically.
The LLM writes code against a domain-specific SDK for:
Defining parts
Composing geometry
Specifying joints
Writing tests to validate the resulting assets
A harness exposes a restricted workspace and interface to the LLM, validates generated assets, and returns structured feedback — freeing the LLM from details like writing URDF files or managing complex software environments.

Results

Generated assets are higher quality than state-of-the-art articulated asset generators and general coding agents.
The authors build Articraft-10K, a curated dataset of over 10K articulated assets spanning 245 categories.
The dataset proves useful for training articulated asset models and for downstream applications such as robot simulation and virtual reality.

Link

Paper: https://arxiv.org/abs/2505.08631