Abstract
Autonomous driving in dense urban environments necessitates a balance between human-like decision-making and rigorous safety constraints. We introduce SHAIL (Safety-Aware Hierarchical Adversarial Imitation Learning), a framework that bridges the gap between adversarial imitation learning and optimization-based safety controllers. SHAIL employs a hierarchical structure where a high-level policy selects from a library of safe low-level "options." By incorporating a safety-aware layer that filters unsafe maneuvers and enables early termination, SHAIL achieves superior performance in complex scenarios. Our evaluation on real-world roundabout data demonstrates that SHAIL significantly improves success rates and robustness compared to standard imitation learning baselines.
Introduction
Developing autonomous vehicles that are both safe and predictable remains a significant challenge. Current methodologies typically rely on either Reinforcement Learning (RL) or Imitation Learning (IL). RL often struggles with "reward misspecification," where agents exploit loopholes in hand-crafted rewards. Conversely, IL methods like Behavior Cloning suffer from "covariate shift," where small execution errors compound, leading the vehicle into unfamiliar and potentially catastrophic states.
While Generative Adversarial Imitation Learning (GAIL) addresses some IL limitations by allowing simulator-based practice, it typically operates on low-level actions (e.g., steering). In high-stakes environments like roundabouts, learning safety from scratch at this level is computationally difficult and prone to failure.
Methodology
SHAIL enhances the standard GAIL framework through two primary innovations:
- Hierarchical Structure: Instead of predicting raw control inputs, the high-level policy selects from a library of predefined "options" (e.g., specific target velocities). These options are executed by fixed, optimization-based low-level controllers designed for safety.
- Safety-Awareness Layer: A safety predictor estimates the probability ($p_{safe}$) of an option's safety. The high-level policy is reweighted to prioritize safe options. Furthermore, if an active option is predicted to become unsafe, control is immediately returned to the high-level policy for a new maneuver (early termination).
The authors reformulate the GAIL objective to match occupancy measures across this hierarchy, enabling the high-level policy to be trained using only low-level expert trajectories without requiring explicit high-level labels.
Experimental Setup
The framework was evaluated using the Interaction Dataset, which contains real-world motion tracks from complex urban scenarios. Testing was conducted in a custom OpenAI Gym-compatible simulator where the "ego" vehicle interacts with other agents following recorded real-world trajectories. The primary test scenario was the "DR_USA_Roundabout_FT" environment.
Results and Discussion
SHAIL was compared against several baselines, including Behavior Cloning (BC), standard GAIL, and an ablation model without the safety layer (HAIL).
As shown in Table 1, SHAIL achieved the highest success rates in both in-distribution (77.7%) and out-of-distribution (64.4%) scenarios. The ablation study (HAIL) confirms that the safety-awareness layer is critical, contributing to a ~25% increase in success rate.
Key Insights
- Safety as a Prerequisite: Treating safety as a filter for action selection effectively mitigates the reward misspecification issues common in RL.
- Generalization: SHAIL demonstrated significantly better robustness to out-of-distribution data than raw end-to-end learning models.
- The Jerkiness Trade-off: While safer, SHAIL exhibited higher acceleration Jensen-Shannon Divergence (JSD) compared to experts. This "jerky" behavior stems from the simplicity of the fixed-acceleration low-level controllers, highlighting a trade-off between safety and human-like comfort.
Conclusion and Future Work
SHAIL successfully integrates safety constraints into an imitation learning framework, providing a robust solution for complex urban navigation. Future research will focus on implementing more sophisticated low-level controllers to improve passenger comfort and exploring the combination of SHAIL with traditional RL rewards to further refine driving behavior beyond the limits of expert demonstrations.
Summary updated