Paper Overview
Field: Computer Vision Authors: Orest Kupyn, Goutam Bhat, Philipp Henzler arXiv: 2506.14703
Summary
Generating explorable 3D scenes from a single image requires strong generative priors and accurate geometric representations suitable for downstream use. Current video diffusion models offer high-quality generation and implicitly encode multi-view geometric structure in latent space. However, existing feedforward latent scene decoders typically output volumetric 3D Gaussians that lack a well-defined surface, limiting their use in simulation or standard graphics pipelines. This motivates decoding surface-aligned primitives that are not only renderable but also closer to explicit geometric assets.
Key Contributions
- FLAT is introduced as the first method to decode triangle splats directly from compressed video diffusion latents in a single pass.
- Predicting planar primitives is notoriously challenging compared to 3D Gaussians due to environmental sensitivity and typically poor gradient flow. FLAT addresses this with two key elements:
- A ray-centric rotation parameterization for triangle regression.
- A novel product window function that improves gradient flow during differentiable triangle rendering.
- On standard benchmarks, FLAT achieves significantly better geometric accuracy than state-of-the-art feedforward baselines while maintaining competitive visual quality.
- A lightweight test-time optimization step converts the predicted triangle set into a fully opaque, game-engine-ready representation supporting real-time rendering.
- By evaluating 3DGS, 2DGS, and triangle splatting variants under the same training setup, the authors provide the first systematic analysis of representation trade-offs in feedforward scene generation.
- arXiv: https://arxiv.org/abs/2506.14703