ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation

Paper Overview

• Field: Computer Vision (CV)
• Authors: Tianjiao Yu, Xinzhuo Li, Yifan Shen
• Published: 2025-07-09
• arXiv: 2507.06842

Summary

Unified 3D foundation models aspire to generate 3D assets and reason about them in language within a single backbone, but their text-3D interaction remains largely implicit. Existing methods concatenate text and 3D tokens into a flat sequence and rely on self-attention, collapsing coarse structural cues and fine geometric details into one undifferentiated representation.

ELSA3D introduces elastic semantic anchoring to structure language and geometric reasoning jointly along matched abstraction scales. Key components include:

• Scale-aware octree tokenizer: represents geometry at multiple abstraction scales.
• Anchor Tokens: sparse cross-modal units that select semantic cues, route them to the most relevant 3D scale, retrieve scale-specific geometric evidence, and write fused signals back into the unified representation—keeping interaction sparse and precise.
• Lightweight per-block router: enables elastic computation and reasoning by selecting which text tokens instantiate anchoring at which geometric scales, concentrating cross-modal capability where alignment is most needed.

Results

ELSA3D achieves state-of-the-art performance on image-to-3D generation, text-to-3D generation, and 3D captioning, outperforming the strongest unified baselines while roughly halving FLOPs and inference latency relative to the non-elastic variant of the same model.