Overview
Field: Computer Vision Authors: Orest Kupyn, Goutam Bhat, Philipp Henzler Published: 2026-06-24 arXiv: 2506.14703
Abstract (translated from Chinese summary)
Generating explorable 3D scenes from a single image requires strong generative priors and accurate geometric representations suitable for downstream use. Current video diffusion models offer high-quality generation and implicitly encode multi-view geometric structure in latent space. However, existing feedforward latent scene decoders typically output volumetric 3D Gaussians that lack a well-defined surface, limiting their use in simulation or standard graphics pipelines. This motivates decoding surface-aligned primitives that are not only renderable but also closer to explicit geometric assets.
The authors ask whether compressed video diffusion latents can be mapped directly to explicit surface primitives in a single pass. To this end, they introduce FLAT and, for the first time, show that triangle splats can be decoded directly from video diffusion latents.
Compared to decoding 3D Gaussians, predicting planar primitives is notoriously more challenging because of their sensitivity to ambient conditions, often leading to poor gradient flow. FLAT solves this with two key elements:
- A ray-centric rotation parameterization for triangle regression
- A novel product window function that improves gradient flow during differentiable triangle rendering
- First method to decode triangle splats directly from video diffusion latents in a single feedforward pass
- Ray-centric rotation parameterization enables stable triangle regression
- Novel product window function improves gradient flow in differentiable triangle rasterization
- Superior geometric accuracy over feedforward baselines at competitive visual quality
- Test-time optimization yields fully opaque, real-time, game-engine-ready assets
- Systematic comparison of 3DGS vs. 2DGS vs. triangle splatting representations
On standard benchmarks, FLAT achieves significantly better geometric accuracy than state-of-the-art feedforward baselines while maintaining competitive visual quality. The authors further demonstrate a lightweight test-time optimization step that converts the predicted triangle sets into fully opaque, game-engine-ready representations with real-time rendering support.
By evaluating 3DGS, 2DGS, and triangle splatting variants under the same training setup, the paper provides the first systematic analysis of representation trade-offs in feedforward scene generation.