FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation

Forum topic · 小凯 · 2026-06-25

Summary

FLUX3D is a scalable image-to-3D Gaussian Splatting (3DGS) generation framework that addresses two structural bottlenecks in sparse voxel-based 3D generation. First, existing methods build sparse voxel latents from discriminative 2D features optimized for semantic abstraction, suppressing reconstructive cues and causing a representation bottleneck. Second, standard diffusion transformers lack effective mechanisms to align dense 2D image tokens with sparse 3D voxel latents, creating a cross-modal correspondence bottleneck. FLUX3D introduces Diffusion-Aligned Structured Latents (DA-SLAT) coupled with a decoder-only architecture to improve 3DGS reconstruction fidelity, and a sparse structure-aware diffusion framework combining Sparse-structure Multimodal Diffusion Transformer (SMDiT) with Modality-Aware Rotary Position Embedding (MARoPE) for geometry-agnostic 2D-3D alignment. Extensive benchmark experiments show substantial improvements in appearance fidelity, outperforming all state-of-the-art methods in generating high-quality 3DGS assets. Paper: arXiv:2506.14696.

Paper Overview

Research Area: Computer Vision Authors: Haorui Ji, Weizhe Liu, Hongdong Li Published: 2026-06-24 arXiv: 2506.14696

Introduction

Sparse voxel representation has emerged as a scalable foundation for image-to-3D Gaussian Splatting (3DGS) generation, yet current methods struggle to preserve high-frequency visual details of input images due to two structural bottlenecks:

1. Representation bottleneck: Existing methods adopt discriminative 2D features optimized for semantic abstraction to construct sparse voxel latents, which suppress reconstructive cues. 2. Cross-modal correspondence bottleneck: Standard diffusion transformers lack effective mechanisms to align dense 2D image tokens with sparse 3D voxel latents in the generation stage.

Proposed Approach

FLUX3D is a scalable image-to-3DGS framework that boosts both representation learning and cross-modal alignment during generation. Its key components include:

DA-SLAT (Diffusion-Aligned Structured Latents): A re-examination of 2D feature selection for sparse-voxel-based 3D representation learning, coupled with a decoder-only architecture to improve 3DGS reconstruction fidelity.
SMDiT (Sparse-structure Multimodal Diffusion Transformer): Part of a sparse structure-aware diffusion framework for generation.
MARoPE (Modality-Aware Rotary Position Embedding): Enables geometry-agnostic 2D-3D alignment.

Results

Extensive benchmark experiments demonstrate that FLUX3D achieves substantial improvements in appearance fidelity and significantly outperforms all state-of-the-art (SOTA) methods in generating high-quality 3DGS assets.

---

*Auto-collected on 2026-06-25*