Paper Overview
Research Area: Computer Vision (CV) Authors: Weiqing Xiao, Hong Li, Xiuyu Yang, Houyuan Chen et al. Release Date: 2026-05-07 arXiv: 2605.06658
Abstract (Translation)
Recent research has shown that large-scale video diffusion models can be repurposed as neural renderers by decomposing video into intrinsic scene representations and forward-rendering them under new lighting. While promising, this paradigm fundamentally relies on accurate intrinsic decomposition, which remains highly unreliable for real-world videos and frequently leads to distorted appearance, broken materials, and accumulated temporal artifacts during relighting.
In this work, we present Relit-LiVE, a novel video relighting framework that produces physically consistent and temporally stable results without requiring camera pose priors. Our core insight is to explicitly introduce the original reference image into the rendering process, allowing the model to recover critical scene cues that are inevitably lost or corrupted in the intrinsic representations.
Furthermore, we propose a novel environment video prediction formulation that simultaneously generates the relit video and per-frame environment maps aligned to each camera viewpoint within a single diffusion process. This joint prediction enforces strong geometry-lighting alignment and naturally supports dynamic lighting and camera motion, significantly improving the physical consistency of video relighting while reducing the requirement for known per-frame camera poses.
Extensive experiments demonstrate that Relit-LiVE consistently outperforms state-of-the-art video relighting and neural rendering methods on both synthetic and real-world benchmarks. Beyond relighting, our framework naturally supports a broad range of downstream applications, including scene-level rendering, material editing, object insertion, and streaming video relighting.
---
*Automatically collected on 2026-05-10.*