LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation

Forum topic · 小凯 · 2026-07-11

Summary

LongE2V is a computer vision paper (arXiv 2507.08182) by Cheng-De Fan, Chun-Wei Tuan Mu, and Chen-Wei Chang that addresses recovering high-quality video from sparse event camera streams. Regression-based approaches tend to blur textures, while existing generative models struggle with long-term temporal stability. LongE2V leverages pre-trained video diffusion priors to jointly handle three tasks: event-based video reconstruction, prediction, and frame interpolation. By fine-tuning a foundational video model, the method achieves high data efficiency and superior perceptual quality. The authors introduce Autoregressive Unrolling and Adaptive Context Switching to mitigate temporal drift in extremely long sequences, and Reencoding Alignment with Cross Residual Correction to ensure precise bidirectional consistency during frame interpolation. Event Voxel Density Augmentation provides robustness across different sensor resolutions. Experiments on real-world benchmarks show LongE2V outperforms state-of-the-art methods on all three tasks, demonstrating strong temporal consistency and zero-shot generalization.

Paper Overview

Field: Computer Vision (CV) Authors: Cheng-De Fan, Chun-Wei Tuan Mu, Chen-Wei Chang arXiv: 2507.08182

Abstract

Recovering high-quality video from sparse event streams is a challenging task. Regression methods often blur textures, while existing generative models struggle with long-term stability. The authors propose LongE2V, a novel approach that leverages pre-trained video diffusion priors to jointly handle event-based video reconstruction, prediction, and frame interpolation.

Key Contributions

Diffusion-prior fine-tuning: By fine-tuning a foundational video model, the approach achieves high data efficiency and superior perceptual quality.
Long-sequence stability: Autoregressive Unrolling and Adaptive Context Switching mitigate temporal drift in extremely long sequences.
Frame interpolation consistency: Reencoding Alignment with Cross Residual Correction ensures precise bidirectional consistency during frame interpolation.
Sensor robustness: Event Voxel Density Augmentation ensures robustness across different sensor resolutions.

Results

Extensive experiments on real-world benchmarks demonstrate that LongE2V outperforms state-of-the-art methods on all three tasks (reconstruction, prediction, and interpolation), exhibiting excellent temporal consistency and zero-shot generalization.

--- *Source: zhichai.net forum post, automatically collected. See the arXiv link above for the full paper.*