TL;DR
CARE-Drive enhances end-to-end autonomous driving by decomposing trajectory refinement into two physically specialized "experts": one for position and one for heading. By training these experts in an alternating, staged curriculum—and conditioning heading on corrected positions—the model achieves a state-of-the-art 88.8 PDMS on the NAVSIM v1 benchmark, significantly outperforming unified refinement approaches.
What Problem Does This Paper Address?
Vision-Language-Action (VLA) models have become a powerhouse in autonomous driving, leveraging massive pre-training to understand complex scenes. However, when it comes to the final "action"—the predicted trajectory—most models treat refinement as a unified black-box task. They might use residual denoising or token correction, but they often ignore the distinct physical roles and interactions between position (where the car is) and heading (which way it’s facing).
The authors argue that a "one-size-fits-all" refinement module struggles to capture the nuances of vehicle dynamics. For example, if a model corrects a car's position to avoid an obstacle, the heading must also be updated to ensure the vehicle is oriented correctly along that new path. CARE-Drive addresses this by introducing physical specialization and conditional coupling into the refinement process.
Methods
The CARE-Drive framework consists of three main components:
- Predictive Backbone: A Qwen3-VL-2B model serves as the world-action backbone. During training, it performs a "future-token auxiliary task," where it learns to predict latent features of future frames (supervised by a frozen V-JEPA 2.1 teacher). This forces the backbone to understand scene evolution, providing a solid base trajectory ($T^{base}$) and shared action features ($H$).
- Specialized Experts: Instead of one large MLP, the authors use two independent 8-layer Transformer decoders:
- Position Expert: Predicts bounded XY residuals to nudge the base trajectory waypoints.
- Heading Expert: Reconstructs orientation from angular increments. Crucially, this expert is conditioned on the corrected positions from the first expert, ensuring the vehicle's "face" follows its "feet."
- Staged Training Curriculum: The experts are optimized with the backbone frozen. The training follows a specific sequence: backbone imitation learning $\rightarrow$ heading supervised fine-tuning (SFT) $\rightarrow$ heading reinforcement learning (RL) $\rightarrow$ position SFT $\rightarrow$ position RL $\rightarrow$ final heading adaptation. They use Group Relative Policy Optimization (GRPO) for the RL stages to maximize planning rewards.
Datasets
The model was evaluated on NAVSIM v1, a popular trajectory-planning benchmark derived from real-world driving logs. It uses a non-reactive simulation environment where the primary metric is the PDM Score (PDMS), a composite score measuring safety (collisions), driving progress, and passenger comfort.
Key Insights and Ideas
- Physical Specialization is Key: The "aha moment" here is that breaking the problem down into physically meaningful sub-tasks (XY vs. Heading) is more effective than simply throwing more parameters at a unified MLP. A parameter-matched MLP expansion only gained 0.048 points, while the dual-expert approach gained 2.375.
- Conditional Coupling: By making the heading expert "aware" of the position expert's corrections, the model maintains geometric consistency. This prevents "sideways sliding" trajectories where the position is right but the orientation is wrong.
- Staged > Joint Training: Surprisingly, training the experts jointly (at the same time) led to significantly worse performance (83.7 PDMS vs. 88.8). The alternating curriculum allows each expert to stabilize before the next layer of complexity is added.
- Future Supervision Matters: Even though the "future teacher" is only used during training, it significantly boosts the backbone's planning ability by 2.49 points, proving that "world modeling" helps "action planning."
Key Results
CARE-Drive achieved 88.837 PDMS on the NAVSIM v1 test set, placing it among the top-performing VLA methods.
- Improvement: It improved the base backbone's performance by 2.4 points.
- Qualitative Gains: As shown in Figure 2, the refinement experts successfully nudged trajectories to avoid "off-road" corners and prevented collisions with stopped cars that the base backbone would have hit.
- Safety Outcomes: The refinement process significantly increased compliance with drivable-area (DA) and time-to-collision (TTC) constraints.
Limitations and Future Work
- Non-Reactive Simulation: NAVSIM v1 is non-reactive, meaning other cars don't respond to the ego-vehicle's actions. Testing in a fully closed-loop, reactive environment is a logical next step.
- Expert Complexity: While position and heading are the most critical, future versions could include experts for velocity profiles or acceleration to further refine comfort and safety.
- Inference Latency: Although the authors argue the latency is manageable, adding multiple autoregressive experts naturally increases the computational cost compared to a single-shot MLP.
For those interested in the implementation, the source code is available at https://github.com/marshal12138/CARE-Drive.
Summary updated