FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation

Forum topic · 小凯 · 2026-06-25

Summary

FLUX3D (arXiv:2506.14696) is a scalable image-to-3D Gaussian Splatting (3DGS) generation framework by Haorui Ji, Weizhe Liu, and Hongdong Li. It addresses two structural bottlenecks in sparse voxel–based 3D generation: discriminative 2D features optimized for semantic abstraction suppress reconstructive cues (representation bottleneck), and standard diffusion transformers lack mechanisms to align dense 2D image tokens with sparse 3D voxel latents (cross-modal correspondence bottleneck). FLUX3D introduces Diffusion-Aligned Structured Latents (DA-SLAT) coupled with a decoder-only architecture for higher reconstruction fidelity, plus a sparse structure–aware diffusion framework built on Sparse structure Multimodal Diffusion Transformer (SMDiT) and Modality-Aware Rotary Position Embedding (MARoPE) for geometry-independent 2D-3D alignment. Benchmarks show substantial improvements in appearance fidelity, outperforming state-of-the-art methods in generating high-quality 3DGS assets.

Overview

Field: Computer Vision Authors: Haorui Ji, Weizhe Liu, Hongdong Li Published: 2026-06-24 arXiv: 2506.14696

Abstract

Sparse voxel representation has emerged as a scalable foundation for image-to-3D Gaussian Splatting (3DGS) generation, yet current methods struggle to preserve high-frequency visual details of input images due to two structural bottlenecks:

1. Representation bottleneck — They adopt discriminative 2D features optimized for semantic abstraction to construct sparse voxel latents, which suppress reconstructive cues. 2. Cross-modal correspondence bottleneck — Standard diffusion transformers lack effective mechanisms to align dense 2D image tokens with sparse 3D voxel latents.

Proposed Solution

To address these issues, the authors propose FLUX3D, a scalable image-to-3DGS framework that boosts both representation learning and cross-modal alignment during generation:

DA-SLAT (Diffusion-Aligned Structured Latents): a re-examination of 2D feature selection for sparse-voxel-based 3D representation learning, coupled with a decoder-only architecture to improve 3DGS reconstruction fidelity.
SMDiT (Sparse structure Multimodal Diffusion Transformer): a sparse structure–aware diffusion framework component.
MARoPE (Modality-Aware Rotary Position Embedding): enables geometry-independent 2D–3D alignment.

Results

Extensive benchmark experiments demonstrate that FLUX3D achieves substantial improvements in appearance fidelity and significantly outperforms all state-of-the-art methods in generating high-quality 3DGS assets.

--- *Auto-collected on 2026-06-25*