Paper Overview
- Field: Computer Vision
- Authors: Weiqing Xiao, Hong Li, Xiuyu Yang
- Published: 2025-05-09
- arXiv: 2505.03481
- Raw reference image injection: Explicitly introducing raw reference images into the rendering process enables the model to recover critical scene cues that are inevitably lost or corrupted in intrinsic representations.
- Joint environment video prediction: A novel formulation generates, in a single diffusion pass, both the relit video and per-frame environment maps aligned with each camera viewpoint. This enforces strong geometry-lighting alignment and naturally supports dynamic lighting and camera motion, improving physical consistency while relaxing per-frame camera pose requirements.
- Scene-level rendering
- Material editing
- Object insertion
- Streaming video relighting
Background
Recent advances have shown that large-scale video diffusion models can be repurposed as neural renderers by first decomposing videos into intrinsic scene representations and then performing forward rendering under novel illumination. While promising, this paradigm fundamentally relies on accurate intrinsic decomposition, which remains highly unreliable for real-world videos and often leads to distorted appearances, broken materials, and accumulated temporal artifacts during relighting.
Proposed Method
Relit-LiVE is a novel video relighting framework that produces physically consistent, temporally stable results without requiring prior knowledge of camera pose. Its key contributions are:
Results
Extensive experiments show that Relit-LiVE consistently outperforms state-of-the-art video relighting and neural rendering methods on both synthetic and real-world benchmarks.
Applications
Beyond relighting, the framework naturally supports a broad range of downstream tasks:
*Auto-collected on 2026-05-09*