ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation

Forum topic · 小凯 · 2026-07-09

Summary

ELSA3D is a unified 3D foundation model (arXiv:2507.06842) that addresses the implicit and coarse text-3D interaction in existing approaches, which concatenate text and 3D tokens into flat sequences and rely on self-attention. ELSA3D introduces elastic semantic anchoring: a scale-aware octree tokenizer represents geometry at multiple abstraction levels, while sparse cross-modal Anchor Tokens select semantic cues, route them to the most relevant 3D scale, retrieve scale-specific geometric evidence, and write fused signals back into the unified representation. Lightweight per-chunk routers decide which text tokens instantiate anchors at which geometric scales, keeping interaction sparse and precise while concentrating cross-modal capability where alignment matters most. ELSA3D achieves state-of-the-art results in image-to-3D generation, text-to-3D generation, and 3D captioning, outperforming the strongest unified baselines while cutting FLOPs and inference latency roughly in half compared to the non-elastic variant of the same model.

Paper Overview

Field: Computer Vision (CV)
Authors: Tianjiao Yu, Xinzhuo Li, Yifan Shen
Released: 2025-07-09
arXiv: 2507.06842

Background

Unified 3D foundation models aim to generate 3D assets and reason about them in language within a single backbone, but their text-3D interaction remains largely implicit. Existing methods concatenate text and 3D tokens into a flat sequence and rely on self-attention, collapsing coarse structural cues and fine geometric details into one undifferentiated representation.

Method: Elastic Semantic Anchoring

ELSA3D structures language and geometric reasoning jointly along matched abstraction scales:

Scale-aware octree tokenizer: represents geometry at multiple abstraction levels, separating structural cues from fine geometric detail.
Anchor Tokens: sparse cross-modal units that select semantic cues, route them to the most relevant 3D scale, retrieve scale-specific geometric evidence, and write the fused signal back into the unified representation — keeping interaction sparse and precise.
Lightweight per-chunk routers: decide which text tokens instantiate anchors at which geometric scales, making computation and reasoning elastic and concentrating cross-modal capability where alignment is needed most.

Results

State-of-the-art performance in image-to-3D generation, text-to-3D generation, and 3D captioning.
Outperforms the strongest unified baselines.
Roughly halves FLOPs and inference latency relative to a non-elastic variant of the same model.