AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving.

Zhenlong Yuan, Jing Tang, Jinguo Luo, Rui Chen, Chengxuan Qian, Lei Sun, Xiangxiang Chu, Yujun Cai, Dapeng Zhang, Shuo LiarXiv 2025 · OpenReview

TL;DR

This paper introduces AutoDrive-R², a novel Vision-Language-Action (VLA) framework for autonomous driving that significantly enhances reasoning and self-reflection capabilities. By employing a chain-of-thought dataset and a physics-grounded reinforcement learning approach, AutoDrive-R² achieves state-of-the-art performance in trajectory planning, producing more reliable and physically feasible driving behaviors.

What Problem Does This Paper Address?

Autonomous driving systems, particularly those using Vision-Language-Action (VLA) models, have shown great promise in integrating perception with decision-making. However, current VLA models often struggle with:

  • Interpretability and Coherence: The decision-making process and the generated action sequences lack transparency and logical consistency.
  • Physical Infeasibility: Trajectory generation can result in physically impossible or unrealistic movements, failing to account for vehicle dynamics and road constraints.
  • Limited Reasoning: Existing models often employ simplistic reasoning strategies, failing to handle complex driving scenarios that require a deeper understanding of context and physics.

These limitations hinder the practical deployment of VLA models in real-world autonomous driving systems.

Methods

AutoDrive-R² employs a two-stage training approach:

  1. Supervised Fine-Tuning (SFT) with Chain-of-Thought (CoT):
    • nuScenesR2-6K Dataset: A novel dataset is introduced, featuring 6,000 image-trajectory pairs enriched with a four-step logical chain for reasoning: Observation, Calculation, Logic, and Self-Reflection. This dataset aims to build cognitive bridges between input information and output trajectories, fostering both reasoning and validation.
  2. CoT Prompt Design: A structured prompt guides the generation of CoT data, breaking down the reasoning process into image-driven analysis, physics-based calculation, contextual logic synthesis, and a crucial self-reflection step for validation.
  3. Reinforcement Learning (RL) with Physics-Grounded Rewards:
    • Group Relative Policy Optimization (GRPO): This algorithm is used for RL training, featuring a pairwise comparison mechanism for candidate responses.
  4. Physics-Grounded Reward Framework: To ensure physically feasible trajectories, the reward function incorporates:
  5. Spatial Alignment: Penalizes deviations from ground-truth trajectories.
  6. Vehicle Dynamics: Enforces constraints on steering angles and velocity to ensure kinematic feasibility.
  7. Temporal Smoothness: Penalizes rapid variations in control signals (steering and velocity) to ensure smooth and predictable motion.

Datasets

  • nuScenesR2-6K: A newly created dataset for supervised fine-tuning, containing 6,000 image-trajectory pairs with detailed chain-of-thought reasoning and self-reflection steps.
  • nuScenes: Used for evaluation, a standard autonomous driving dataset with synchronized camera views.
  • Waymo: Also used for evaluation, another comprehensive autonomous driving dataset.

Key Insights and Ideas

  • The Power of Self-Reflection: The four-step CoT process, particularly the self-reflection stage, is crucial. It allows the model to identify and correct inconsistencies in its reasoning, leading to more robust and accurate predictions. This emergent "reasoning self-correction moment" is a key differentiator.
  • Bridging Perception and Physics: Simply predicting trajectories based on visual and linguistic input is insufficient. Integrating physics-grounded rewards (spatial alignment, vehicle dynamics, temporal smoothness) is essential for generating trajectories that are not only accurate but also physically executable and comfortable for passengers.
  • CoT for Foundational Reasoning: The nuScenesR2-6K dataset, with its structured reasoning and self-reflection, provides a strong foundation for VLA models, significantly improving their ability to handle complex driving scenarios before RL fine-tuning.
  • Two-Stage Training is Key: The combination of SFT with CoT data followed by physics-grounded RL is more effective than either approach alone, addressing both reasoning and physical feasibility challenges.

Key Results

  • State-of-the-Art Performance: AutoDrive-R² consistently outperforms leading methods on both nuScenes and Waymo datasets, achieving significantly lower L2 errors and collision rates.
  • Significant Improvement over Baselines: On nuScenes, AutoDrive-R² reduces L2 errors by 86.9% compared to the Qwen2-VL-7B baseline, despite using a much smaller training dataset.
  • Robust Generalization: The model demonstrates strong zero-shot capabilities on the Waymo dataset, outperforming advanced methods like EMMA+ and Qwen2-VL-72B.
  • Improved Safety and Quality: In closed-loop experiments on the NAVSIM benchmark, AutoDrive-R² achieves top scores in safety metrics (Navigation Completion, Time-to-Collision) and driving quality (Driving Agent Comfort, Ego-Progress).

Limitations and Future Work

  • Model Size Impact: While the 7B model achieves superior performance, the 3B variant also shows notable improvement over its baseline, suggesting the two-stage framework effectively compensates for smaller model capacities. However, larger models inherently capture more complex patterns.
  • Future Work: The authors suggest future efforts will focus on multi-agent coordination and real-time sensor fusion integration to further enhance adaptability in complicated environments.

Summary updated

BibTeX
@misc{yuan2025autodrive,
  author = {Zhenlong Yuan and Jing Tang and Jinguo Luo and Rui Chen and Chengxuan Qian and Lei Sun and Xiangxiang Chu and Yujun Cai and Dapeng Zhang and Shuo Li},
  title = {{AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving.}},
  year = {2025},
  url = {https://openreview.net/forum?id=eWWgQqdkta},
}
Figure