Abstract
Most end-to-end autonomous driving methods rely on imitation learning from single expert demonstrations, often leading to conservative and homogeneous behaviors that limit generalization in complex real-world scenarios. In this work, we propose DIVER, an end-to-end driving framework that integrates reinforcement learning with diffusion-based generation to produce diverse and feasible trajectories. At the core of DIVER lies a reinforced diffusion-based generation mechanism. First, the model conditions on map elements and surrounding agents to generate multiple reference trajectories from a single ground-truth trajectory, alleviating the limitations of imitation learning that arise from relying solely on single expert demonstrations. Second, reinforcement learning is employed to guide the diffusion process, where reward-based supervision enforces safety and diversity constraints on the generated trajectories, thereby enhancing their practicality and generalization capability. Furthermore, to address the limitations of L2-based open-loop metrics in capturing trajectory diversity, we propose a novel Diversity metric to evaluate the diversity of multi-mode predictions. Extensive experiments on the closed-loop NAVSIM and Bench2Drive benchmarks, as well as the open-loop nuScenes dataset, demonstrate that DIVER significantly improves trajectory diversity, effectively addressing the mode collapse problem inherent in imitation learning.
Introduction
Current end-to-end autonomous driving (E2E-AD) systems predominantly utilize Imitation Learning (IL), training models to minimize the distance between predicted paths and a single "ground truth" (GT) expert trajectory. While this approach works in simple environments, it creates a significant "diversity bottleneck." Because models are penalized for deviating from a single path, they often suffer from mode collapse, failing to account for multiple valid maneuvers in complex scenarios like intersections. This results in overly conservative, "robotic" driving behaviors that struggle to generalize.
DIVER (Reinforced Diffusion) addresses this by treating trajectory planning as a generative task guided by task-specific rewards, moving beyond the constraints of single-mode imitation.
Methodology: The DIVER Framework
DIVER introduces two primary components to overcome the imitation bottleneck: the Policy-Aware Diffusion Generator (PADG) and Reinforced Diffusion via Group Relative Policy Optimization (GRPO).
Policy-Aware Diffusion Generator (PADG)
The PADG is a conditional diffusion model designed to reconstruct future trajectories from random noise. Unlike standard planners that rely on a single GT, PADG conditions on rich scene semantics, including BEV maps and agent interactions, alongside multiple reference GTs. This allows the model to learn a broader distribution of human-like behaviors. A specialized TrajPooler module is used to focus perception features on regions relevant to the predicted paths, ensuring high-fidelity environmental awareness.
Reinforced Diffusion via GRPO
To ensure that the generative power of the diffusion model translates into safe and effective driving, the authors treat the diffusion process as a stochastic policy and fine-tune it using GRPO. This reinforcement learning stage optimizes for non-differentiable objectives that standard IL cannot capture:
- Diversity Reward ($r_{div}$): Explicitly penalizes trajectories that are too similar, encouraging the exploration of multiple valid paths.
- Safety Reward ($r_{safe}$): Penalizes proximity to obstacles and potential collisions.
- Consistency and Lane Keeping: Ensures smooth, rule-abiding trajectories.
Experimental Results
The researchers evaluated DIVER across several benchmarks, demonstrating its superiority in both open-loop and closed-loop settings.
Performance on Benchmarks
- Bench2Drive (Closed-Loop): DIVER achieved a 29.0% improvement in Success Rate and a 10.5% increase in Driving Score over the previous state-of-the-art, SparseDrive.
- NAVSIM: The model reached a top PDM Score of 88.3, highlighting its excellence in collision avoidance and lane compliance.
- nuScenes (Open-Loop): DIVER increased the Diversity Metric by approximately 61.5% compared to DiffusionDrive, while simultaneously lowering collision rates.
Robustness and Diversity
In adversarial scenarios (Adv-nuSc), DIVER maintained a collision rate of only 0.752%, significantly lower than UniAD (3.95%) and VAD (7.05%). The authors also introduced a new Diversity Metric (Div.) to better quantify multi-mode planning performance, as traditional L2 metrics often fail to capture the value of exploring multiple valid solutions.
Conclusion and Future Work
DIVER effectively breaks the imitation bottleneck by combining the generative flexibility of diffusion models with the goal-oriented supervision of reinforcement learning. While it excels at generating a diverse set of feasible trajectories, the challenge of optimal trajectory selection remains. Future research may involve integrating Vision-Language-Action (VLA) models to leverage high-level reasoning for selecting the most appropriate path from the generated diverse set.
Summary updated