Paper Overview
- Field: Computer Vision (CV)
- Authors: Jiahao Wang, Bo Sun, Yijing Bai
- Release Date: 2026-05-25
- arXiv: 2505.14490
- Problem addressed: A mismatch between the abundance of unstructured monocular dashcam video and the structured, multi-modal sensor inputs required for ADS training and validation.
- Core technical idea: Convert real AV logs into dashcam-style video through 4D Gaussian Splatting (4DGS) reconstruction plus novel view rendering, producing the missing paired data needed for supervised training.
- Conversion model: A diffusion-based architecture learns the cross-embodiment translation from dashcam-style monocular input to a full multi-modal sensor suite (multi-view cameras + LiDAR).
- Evaluation: A comprehensive quantitative assessment of the fidelity and realism of the generated sensor data, including comparisons against standard metrics.
- Demonstrated utility: Challenging wild internet and dashcam clips are converted into realistic multi-modal AV-log-style outputs, illustrating how large external video sources can be unlocked for ADS development.
Summary
Robust training and validation of autonomous driving systems (ADS) requires large-scale, diverse datasets. Proprietary data collected by AV fleets is high-fidelity but limited in scale, sensor-configuration variety, and geographic and long-tail behavioral coverage. By contrast, wild data—e.g., dashcam videos—offers far greater volume and diversity, capturing critical long-tail scenarios and new environments. However, such unstructured wild video is incompatible with ADS, which expect structured, multi-modal sensor inputs. To bridge this data gap, the authors propose Sensor2Sensor, a novel generative modeling paradigm that translates wild monocular dashcam video into a high-fidelity, multi-modal sensor suite (an "AV log") comprising multi-view camera images and LiDAR point clouds.
Key Points
Source
Full Chinese abstract preserved below for reference:
> 为弥合这一数据鸿沟，我们提出Sensor2Sensor，一种新颖的生成建模范式，将野生单目行车记录仪视频转换为高保真、多模态传感器套件（AV日志），包括多视角相机图像和LiDAR点云。核心挑战在于缺乏配对训练数据。我们通过4D高斯溅射（4DGS）重建和新视角渲染将真实AV日志转换为行车记录仪风格视频来解决这一问题。Sensor2Sensor然后利用扩散架构进行生成转换。我们对生成传感器数据的保真度和真实性进行了全面的定量评估。我们通过将具有挑战性的野生互联网和行车记录仪片段转换为逼真的多模态数据格式，展示了Sensor2Sensor的实用价值，进一步解锁了海量外部数据源用于自动驾驶开发。
*Auto-collected 2026-05-25.*