PAR3D: A Unified Part-Aware 3D-MLLM for Scene Understanding

Forum topic · 小凯 · 2026-06-07

Summary

PAR3D is a unified 3D multimodal large language model (3D-MLLM) framework introduced by Shaohui Dai, Yansong Qu, and You Shen (arXiv:2606.06485) that extends 3D scene understanding beyond the object level to fine-grained part-level reasoning. While existing 3D-MLLMs handle visual question answering, captioning, and referring segmentation, they remain largely object-centric, limiting embodied interaction with 3D environments. PAR3D enables models to understand, reason about, and ground both objects and their constituent parts in 3D scenes. The authors also release ScenePart, a synthetic 3D scene dataset with part-level annotations and language instructions for training and evaluation. The approach features part-aware 3D representation learning that enriches visual features with fine-grained part-level semantics, and hierarchical segmentation query generation that localizes part-level targets through hierarchical object-part queries. Experiments show significant gains in part-level question answering and referring segmentation, while maintaining strong performance on object-level vision-language tasks.

Paper Overview

Field: Computer Vision Authors: Shaohui Dai, Yansong Qu, You Shen arXiv: 2606.06485

Introduction

Recent advances in 3D multimodal large language models (3D-MLLMs) have enabled unified solutions for 3D scene understanding tasks, including visual question answering, captioning, and referring segmentation. However, existing 3D-MLLMs remain largely object-centric, limiting their ability to model fine-grained part structures that are essential for embodied interaction with 3D environments.

Key Contributions

PAR3D Framework: A unified part-aware 3D-MLLM that enables models to understand, reason about, and ground both objects and their parts in 3D scenes.
ScenePart Dataset: A synthetic 3D scene dataset with part-level annotations and language instructions, designed to support training and evaluation of part-aware 3D scene understanding.
Part-Aware 3D Representation Learning: Enriches 3D visual representations with fine-grained part-level semantics.
Hierarchical Segmentation Query Generation: Localizes part-level targets through hierarchical object-part queries.

Results

Extensive experiments demonstrate that PAR3D significantly improves part-level question answering and referring segmentation, while also achieving strong performance on object-level vision-language tasks.