$AutoDrive\text{-}P^3$: Unified Chain of Perception–Prediction–Planning Thought via Reinforcement Fine-Tuning

TL;DR

$AutoDrive\text{-}P^3$ is an end-to-end autonomous driving framework that unifies Perception, Prediction, and Planning into a single, interpretable Chain-of-Thought (CoT) using Vision-Language Models (VLMs). By introducing a hierarchical reinforcement learning algorithm ($P^3\text{-}GRPO$) and a specialized reasoning dataset, the system achieves state-of-the-art performance on major benchmarks, significantly reducing collision rates while providing human-readable rationales for its actions.

---

What Problem Does This Paper Address?

While Vision-Language Models (VLMs) are exceptionally good at "understanding" the world, applying them to autonomous driving has hit two major roadblocks:

1. The "Black Box" Problem: Many models jump straight from images to a driving trajectory. This lacks transparency and skips the intermediate reasoning (like identifying a pedestrian or predicting their path) that human drivers naturally use.
2. Task Fragmentation: Models that do handle perception and prediction often treat them as isolated sub-tasks. This lack of synergy means the "planning" module doesn't fully benefit from the "perception" module's insights, leading to poor performance in complex, "long-tail" scenarios.

Expand

Source, p. 2

The authors aim to bridge this gap by forcing the model to "think" through the stages of driving in a structured, interdependent sequence, moving away from fragmented or purely black-box paradigms.

Methods

The authors propose a three-pronged approach to unify the driving pipeline:

• Structured Chain-of-Thought (CoT): Instead of a single output, the model generates a sequential reasoning chain: <perception_thinking> $\rightarrow$ <perception_answer> $\rightarrow$ <prediction_thinking> $\rightarrow$ <prediction_answer> $\rightarrow$ <planning>. This ensures that the final plan is grounded in the model's own perception and prediction logic.
• $P^3\text{-}GRPO$ (Hierarchical Reinforcement Learning): Building on the Group Relative Policy Optimization (GRPO) algorithm, the authors introduce a hierarchical reward system. Instead of just rewarding a good final trajectory, they provide "progressive supervision" with specific rewards for accurate object detection (Perception) and behavior forecasting (Prediction).
• Dual Thinking Modes: To address the high computational cost of VLMs, the framework supports a Detailed Thinking mode (full reasoning for maximum safety and interpretability) and a Fast Thinking mode (skipping the reasoning text to achieve near real-time 1Hz inference).

Datasets

The researchers introduced the $P^3\text{-}CoT$ dataset, which provides unified reasoning chains for:

• nuScenes: 25,303 frames from 850 scenes.
• NAVSIM: 115,434 frames from 1,382 scenes.

Unlike previous datasets that offered fragmented Q&A pairs, $P^3\text{-}CoT$ connects perception, prediction, and planning into a single coherent narrative for every frame.

Key Insights and Ideas

• Synergy is Measurable: The paper demonstrates that rewarding a model for better perception and prediction directly leads to better planning. It’s not just about the final coordinates; the logic used to get there matters.
• RL over SFT: While Supervised Fine-Tuning (SFT) helps the model learn the format of driving, Reinforcement Learning ($P^3\text{-}GRPO$) is what actually improves the quality of the driving by allowing the model to explore and self-correct its reasoning paths.
• Interpretability Reduces Error: By forcing the model to verbalize its perception (e.g., "I see a truck on the right"), the system reduces the "domain gap" between abstract visual features and concrete driving commands.

Key Results

The framework was tested on both open-loop (nuScenes) and closed-loop (NAVSIM) benchmarks:

• Safety: On nuScenes, $AutoDrive\text{-}P^3$ reduced the collision rate by 40% compared to previous state-of-the-art (SOTA) methods.
• Planning Accuracy: It achieved a SOTA average L2 displacement error of 0.33m on nuScenes.
• Benchmark Dominance: On the NAVSIM v1 benchmark, it achieved a PDMS score of 90.6, outperforming much larger models (like LLaVA-7B) despite using a more compact Qwen2.5-3B backbone.
• Efficiency: The "Fast" mode achieved near real-time performance (1 Hz) on an H100 GPU, a significant feat for a VLM-based end-to-end system.

Limitations and Future Work

• Hallucinations: Like all LLMs, the model can occasionally "hallucinate" objects or reasoning steps that don't exist in the visual data.
• Offline Training: The current reinforcement learning is conducted in offline simulators. The authors suggest that moving to online, interactive environments would further improve robustness.
• Inference Latency: While 1 Hz is fast for a VLM, it is still slow compared to traditional non-VLM autonomous systems. Future work will focus on further reducing inference time and mitigating hallucinations.