[论文] [论文] DreamStream: Towards Policy-Oriented Generative Simulation f...

论文概要

研究领域: CV 作者: Ziyang Leng, Sicheng Mo, Seth Z. Zhao, Haoyuan Cai, Yu Zeng et al. 发布时间: 2026-09-22 arXiv: 2609.26792

中文摘要

在仿真中真实评估端到端驾驶策略，要求观测不仅照片级真实，更要保留策略用于决策的场景特征。现有平台存在 sim-to-real 视觉差距，污染策略感知，削弱其评估闭环决策的能力。为此提出 DreamStream——生成式闭环仿真器，借仿真器接地的自回归视频模型实现"面向策略的保真"。视频模型由大型预训练视频模型经交通布局引导蒸馏而来，改变视觉外观的同时保留策略相关特征（场景布局、动态物体时间一致性）。我们进一步发现 FID 等感知指标会错排这些特征的保留度，故引入 FDπ——在公开 E2E 策略的场景上下文特征上以弗雷歇距离度量 sim-to-real 差距。FDπ 下 DreamStream 在 nuScenes 上较最强闭环仿真器提升 1.6×，NAVSIM 上 4.7×，对策略感知扰动最小。基于 DreamStream 构建 Navhard-CL 基准，把非反应式真实基准 NAVSIM 变为带对抗驾驶行为与天气变化的交互测试环境，暴露评分器偏差、缺乏恢复行为等此前闭环基准忽视的失效模式。代码与数据开源于 https://github.com/VAIL-UCLA/DreamStream。

原文摘要

Faithfully evaluating end-to-end driving policies in simulation requires observations that are not merely photo-realistic, but preserve the scene features a policy relies on to make decisions. Existing platforms, however, exhibit a sim-to-real visual gap that corrupts policy perception, undermining their ability to assess a policy's closed-loop decision-making. To this end, we propose DreamStream, a generative, closed-loop simulator that achieves policy-oriented fidelity using a simulator-grounded autoregressive video model. Our video model is distilled from a large pretrained video model via traffic layout guidance, varying visual appearance while preserving policy-relevant features such as scenario layout and the temporal consistency of dynamic objects. We further observe that perceptual metrics like FID misrank how well these features are preserved. To tackle this, we introduce FD\(π\), a new multi-representation metric that measures the sim-to-real gap as the Fréchet distance over scene-context features from public E2E policies. Under FD\(π\), DreamStream improves over the strongest prior closed-loop simulator by \(1.6\times\) on nuScenes and \(4.7\times\) on NAVSIM, and induces the least perturbation to policy's perceptual observability. Based on DreamStream, we construct Navhard-CL benchmark, which turns non-reactive real-world benchmark NAVSIM into interactive testing environments with adversarial driving behaviors and weather variations. This benchmark exposes many failure modes of driving policies, such as scorer bias and lack of recovery behaviors, that prior closed-loop benchmarks overlook. Code and data are available at https://github.com/VAIL-UCLA/DreamStream.

---

*自动采集于 2026-09-24*

#论文 #arXiv #CV #小凯