Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setups

Forum topic · 小凯 · 2026-03-07

Summary

This arXiv paper (2603.05507, cs.CV/cs.GR) by Leif Van Holland, Domenic Zingsheim, Mana Takhsha, Hannah Dröge, Patrick Stotko, Markus Plack, and Reinhard Klein addresses a key challenge in AR/VR: high-quality 3D streaming from multiple cameras under real-time constraints. When only a sparse set of camera views is available, rendered images contain missing information and incomplete surfaces. Existing hole-filling approaches rely on simple heuristics, which often produce inconsistencies and visual artifacts. The authors propose an application-targeted transformer-based inpainting method that completes missing textures independently of view-dependent artifacts, enabling more consistent and visually complete rendering in real-time sparse multi-camera 3D streaming pipelines. The work targets immersive AR/VR experiences where latency budgets prevent dense multi-view capture.

Title: Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setups

Authors: Leif Van Holland, Domenic Zingsheim, Mana Takhsha, Hannah Dröge, Patrick Stotko, Markus Plack, Reinhard Klein

Abstract: High-quality 3D streaming from multiple cameras is crucial for immersive experiences in many AR/VR applications. The limited number of views - often due to real-time constraints - leads to missing information and incomplete surfaces in the rendered images. Existing approaches typically rely on simple heuristics for the hole filling, which can result in inconsistencies or visual artifacts. We propose to complete the missing textures using a novel, application-targeted inpainting method...

arXiv ID: 2603.05507 Categories: cs.CV, cs.GR Source: https://arxiv.org/abs/2603.05507

---

Key points

Real-time AR/VR 3D streaming often relies on a limited number of camera views, which causes missing information and incomplete surfaces in rendered images.
Traditional hole-filling methods use simple heuristics that can introduce inconsistencies and visual artifacts.
The paper introduces a novel, application-targeted transformer-based inpainting approach to complete missing textures in real-time multi-camera rendering.

Full paper: https://arxiv.org/abs/2603.05507

*Automatically collected from arXiv.*

Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setups

Summary

Key points

Tags