SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesis

Forum topic · 小凯 · 2026-08-19

Summary

SplatGuide is a pose-free novel view synthesis framework that combines feed-forward 3D Gaussian Splatting (3DGS) reconstruction with multi-view diffusion. Unlike prior pipelines that extract only a single signal from the reconstruction (pixel rendering or learned features), SplatGuide reuses one 3DGS scene across three complementary roles: rendered images provide pixel-aligned geometric conditioning; per-Gaussian source-view indices are rendered into a target-view voting map enabling occlusion-aware reference selection; and reconstruction tokens supply feature-level guidance via cross-attention. All three signals come from a single reconstruction forward pass. Evaluated on RealEstate10K, DL3DV, Tanks-and-Temples, and Mip-NeRF 360, SplatGuide achieves state-of-the-art pose-free novel view synthesis, and on RealEstate10K with a moderate number of input views it surpasses baselines that use ground-truth poses. Paper: arXiv 2608.16863.

Paper Overview

Field: Computer Vision Authors: Yejun Zhang, Zihan Wang, Xu Ji et al. (11 authors) arXiv: 2608.16863

Abstract (English)

Generating photorealistic novel views from unposed images requires both 3D geometric understanding and the ability to synthesize unseen content. A natural strategy combines feed-forward 3DGS reconstruction with multi-view diffusion. Yet prior pipelines extract at most one signal from the reconstruction, either pixel rendering or learned features, while none exploits per-Gaussian visibility for occlusion-aware reference selection. This information disconnect leaves renderable geometry, visibility cues, and learned features unused.

SplatGuide closes this disconnect by reusing a single 3DGS scene across three complementary roles:

Pixel-aligned geometric conditioning: rendered images from the 3DGS reconstruction guide the diffusion process at the pixel level.
Occlusion-aware reference selection: per-Gaussian source-view indices are rendered into a target-view voting map to choose the best references under occlusion.
Feature-level guidance: reconstruction tokens provide learned feature guidance through cross-attention.

All three signals are derived from a single reconstruction forward pass.

Results

Experiments on RealEstate10K, DL3DV, Tanks-and-Temples, and Mip-NeRF 360 show that SplatGuide achieves state-of-the-art pose-free novel view synthesis. Notably, on RealEstate10K with a moderate number of input views, it surpasses baselines that use ground-truth poses.

---

*Auto-collected on 2026-08-19.*