ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation (arXiv 2507.06842)

Forum topic · 小凯 · 2026-07-09

Summary

ELSA3D is a unified 3D foundation model introduced by Tianjiao Yu, Xinzhuo Li, and Yifan Shen (arXiv:2507.06842, July 2025) that improves text-3D interaction through elastic semantic anchoring. Unlike existing approaches that concatenate text and 3D tokens into a flat sequence and rely on self-attention—mixing coarse structural cues with fine geometric details—ELSA3D aligns language and geometric reasoning along matched abstraction scales. It represents geometry with a scale-aware octree tokenizer and introduces Anchor Tokens: sparse cross-modal units that select semantic cues, route them to the most relevant 3D scale, retrieve scale-specific geometric evidence, and write fused signals back into the unified representation. Lightweight per-block routers decide which text tokens instantiate anchoring at which geometric scales, keeping computation elastic and concentrating cross-modal capability where alignment matters most. ELSA3D achieves state-of-the-art results in image-to-3D generation, text-to-3D generation, and 3D captioning, outperforming the strongest unified baselines while roughly halving FLOPs and inference latency compared with the non-elastic variant of the same model.

Paper Overview

Field: Computer Vision (CV)
Authors: Tianjiao Yu, Xinzhuo Li, Yifan Shen
Published: 2025-07-09
arXiv: 2507.06842

Abstract

Unified 3D foundation models aspire to generate 3D assets and reason about them in language within a single backbone, but their text-3D interaction remains largely implicit. Existing methods concatenate text and 3D tokens into a flat sequence and rely on self-attention, collapsing coarse structural cues and fine geometric details into one undifferentiated representation.

We introduce ELSA3D, a unified 3D model that addresses this with elastic semantic anchoring, structuring language and geometric reasoning jointly along matched abstraction scales.

Method

Scale-aware octree tokenizer: represents geometry at multiple abstraction scales.
Anchor Tokens: sparse cross-modal units that select semantic cues, route them to the most relevant 3D scale, retrieve scale-specific geometric evidence, and write the fused signal back into the unified representation — keeping interactions sparse and precise.
Lightweight per-block routers: decide which text tokens instantiate anchoring at which geometric scales, concentrating cross-modal capability where alignment is needed most and making computation and inference elastic.

Results

ELSA3D achieves state-of-the-art performance on:

Image-to-3D generation
Text-to-3D generation
3D captioning

It outperforms the strongest unified baselines while roughly halving FLOPs and inference latency relative to the non-elastic variant of the same model.

---

*Auto-collected on 2026-07-09*