TL;DR
Autonomous driving systems struggle with rare but critical safety scenarios. This paper introduces CurricuVLM, a novel framework that uses Vision-Language Models (VLMs) to dynamically generate personalized training curricula, adapting to an AV's learning progress and specifically targeting its weaknesses to improve safety.
What Problem Does This Paper Address?
Ensuring the safety of autonomous vehicles (AVs) is paramount, especially when encountering rare, high-consequence "safety-critical" events. While researchers can generate these scenarios, effectively integrating them into the training process to improve AV policy learning has been a challenge. Existing methods often use static sets of pre-generated scenarios, which become less effective as the AV learns and its capabilities evolve. Furthermore, creating personalized training curricula that adapt to an individual AV's specific performance bottlenecks and learning progress remains largely unexplored. This paper aims to bridge this gap by developing a system that can analyze AV behavior in safety-critical situations and dynamically generate tailored training experiences.
Methods
CurricuVLM employs a closed-loop learning system with three core mechanisms:
- Safety-Critical Event Analysis with Vision-Language Models (VLMs): When an AV encounters a safety-critical event during training, its visual observations are fed into a VLM (specifically GPT-40 in this work). The VLM generates detailed textual descriptions of the event, including the nature of the violation, object positioning, AV responses, and contextual factors.
- Behavioral Pattern Analysis and Recommendation: These VLM-generated descriptions are then processed by GPT-40, guided by a structured prompt. This analysis identifies recurring behavioral patterns, causal factors for failures, and specific areas of weakness. The output includes a "Situation," "Reasoning," "Recommendation," and "CriticalArea."
- Personalized Curriculum Adaptation via Scenario Generation: Based on the insights from GPT-40, the framework dynamically generates personalized safety-critical scenarios. This involves predicting background vehicle (BV) trajectories using a pre-trained model (DenseTNT) and then optimizing BV trajectories to create situations that specifically target the AV's identified weaknesses. This is formulated as a conditional trajectory generation problem.
- Dynamic Curriculum Scheduling: A probability-based mechanism adaptively integrates these personalized safety-critical scenarios into the training curriculum alongside regular scenarios. This ensures a progressive increase in difficulty and exposure to critical events as the AV learns.
Datasets
The experiments were conducted using the Waymo Open Motion Dataset. This dataset provides real-world driving scenarios with detailed trajectory information for ego vehicles and surrounding traffic participants, along with HD map annotations. The authors used 500 representative scenarios, splitting them into 400 for training and 100 for testing.
Key Insights and Ideas
- VLMs as "Observant Mentors": The core innovation is using VLMs not just for scene understanding but as active participants in the learning process, akin to a human instructor analyzing mistakes and designing targeted exercises.
- Personalization is Key: Static curricula are insufficient. Dynamically adapting training scenarios to an AV's specific failure modes and evolving capabilities is crucial for robust safety.
- Bridging the Gap: CurricuVLM effectively bridges the gap between scenario generation and policy learning by creating a feedback loop where observed failures directly inform future training.
- Interpretable Analysis: The VLM-based analysis provides human-understandable insights into why an AV fails, which is a significant advantage over black-box approaches.
Key Results
CurricuVLM demonstrated superior performance across various metrics compared to state-of-the-art baselines (including traditional RL, safe RL, imitation learning, and other curriculum learning methods) on both regular and safety-critical test scenarios.
- Regular Scenarios: Achieved higher episode rewards (52.3 vs. 49.3 for TD3) and road completion rates (76.2% vs. 72.9% for TD3), while maintaining a competitive crash rate (16.0%).
- Safety-Critical Scenarios: Showed significant improvements, with a substantially lower crash rate (25.1% vs. 39.7% for TD3), higher episode rewards (48.9 vs. 42.4 for TD3), and superior road completion rates (73.4% vs. 65.2% for TD3).
- Learning Effectiveness: Achieved a significantly higher failure-to-success rate (39.1% vs. 28.6% for TD3), indicating better learning from mistakes.
- Algorithm Agnostic: The framework showed compatibility with various RL algorithms (TD3, SAC, PPO), suggesting its general applicability.
Limitations and Future Work
- VLM Choice Sensitivity: The performance can be sensitive to the choice of VLM. While GPT-40 and Llama-3.2-11B-Vision performed well, others showed limitations.
- Batch Size Impact: The batch size for VLM analysis needs careful tuning; too small leads to conservative behavior, too large delays analysis.
- Future Work:
- Fine-tuning VLMs on autonomous driving data for improved understanding.
- Incorporating Retrieval-Augmented Generation (RAG) to reference traffic rules and guidelines.
- Developing methods to directly generate realistic scenarios from VLM natural language recommendations.
Summary updated