RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework for Closed-Loop Autonomous Driving Planning

Forum topic · 小凯 · 2026-04-18

Summary

RAD-2 (arXiv:2504.13094) is a unified generator-discriminator framework for closed-loop motion planning in autonomous driving. A diffusion-based generator produces diverse trajectory candidates, while an RL-optimized discriminator reranks them by long-term driving quality. This decoupled design avoids applying sparse scalar rewards directly to high-dimensional trajectory spaces, improving optimization stability over imitation-only diffusion planners that suffer stochastic instability and lack corrective negative feedback. The method introduces Temporally Consistent Group Relative Policy Optimization to mitigate credit assignment via temporal coherence, and On-policy Generator Optimization that converts closed-loop feedback into structured longitudinal optimization signals, steering the generator toward high-reward trajectory manifolds. To enable efficient large-scale training, the authors present BEV-Warp, a high-throughput simulation environment that performs closed-loop evaluation directly in bird's-eye-view feature space via spatial warping. RAD-2 reduces collision rate by 56% compared to strong diffusion-based planners, and real-world deployment demonstrates improved perceived safety and driving smoothness in complex urban traffic.

Paper Overview

Field: Computer Vision / Autonomous Driving
Authors: Hao Gao, Shaoyu Chen, Yifan Zhu
Published: 2025-04-17
arXiv: 2504.13094

Introduction

Advanced autonomous driving requires motion planners that can model multimodal future uncertainty while staying robust in closed-loop interactions. Diffusion-based planners are effective at modeling complex trajectory distributions, but when trained purely with imitation learning they suffer from stochastic instability and lack corrective negative feedback. RAD-2 addresses these problems with a unified generator-discriminator framework for closed-loop planning.

Method

Diffusion-based generator: produces diverse trajectory candidates.
RL-optimized discriminator: reranks candidates according to long-term driving quality. This decoupled design avoids directly applying sparse scalar rewards to the full high-dimensional trajectory space, improving optimization stability.
Temporally Consistent Group Relative Policy Optimization (TC-GRPO): exploits temporal coherence to mitigate the credit assignment problem in RL.
On-policy Generator Optimization: converts closed-loop feedback into structured longitudinal optimization signals, gradually steering the generator toward high-reward trajectory manifolds.
BEV-Warp: a high-throughput simulation environment that performs closed-loop evaluation directly in bird's-eye-view feature space via spatial warping, supporting efficient large-scale training.

Results

Reduces collision rate by 56% compared to strong diffusion-based planners.
Real-world deployment shows improved perceived safety and driving smoothness in complex urban traffic.