Paper Overview
Research Area: Computer Vision (CV) Authors: Weiqing Xiao, Hong Li, Xiuyu Yang, Houyuan Chen et al. Published: 2026-05-07 arXiv: 2605.06658
Abstract (translated)
Recent studies have shown that large-scale video diffusion models can be repurposed as neural renderers by decomposing video into intrinsic scene representations and forward-rendering under new lighting. While promising, this paradigm fundamentally relies on accurate intrinsic decomposition, which remains highly unreliable for real-world videos, often leading to distorted appearance, broken materials, and accumulated temporal artifacts during relighting.
In this work, the authors propose Relit-LiVE, a novel video relighting framework that produces physically consistent, temporally stable results without requiring camera pose priors. The core insight is to explicitly introduce the original reference image into the rendering process, enabling the model to recover critical scene cues that are inevitably lost or corrupted in intrinsic representations.
Additionally, a novel environment video prediction formulation is proposed, which simultaneously generates the relit video and per-frame environment maps aligned with each camera viewpoint in a single diffusion process. This joint prediction enforces strong geometry-lighting alignment and naturally supports dynamic lighting and camera motion, significantly improving the physical consistency of video relighting while reducing the requirement for known per-frame camera poses.
Extensive experiments show that Relit-LiVE consistently outperforms state-of-the-art video relighting and neural rendering methods on both synthetic and real-world benchmarks. Beyond relighting, the framework naturally supports a wide range of downstream applications, including scene-level rendering, material editing, object insertion, and streaming video relighting.
---
*Auto-collected on 2026-05-10.*