SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments

Arec L. Jamgochian, Etienne Bührle, Johannes Fischer, Mykel J. Kochenderfer

Abstract

Autonomous driving in dense urban environments necessitates a balance between human-like decision-making and rigorous safety constraints. We introduce SHAIL (Safety-Aware Hierarchical Adversarial Imitation Learning), a framework that bridges the gap between adversarial imitation learning and optimization-based safety controllers. SHAIL employs a hierarchical structure where a high-level policy selects from a library of safe low-level "options." By incorporating a safety-aware layer that filters unsafe maneuvers and enables early termination, SHAIL achieves superior performance in complex scenarios. Our evaluation on real-world roundabout data demonstrates that SHAIL significantly improves success rates and robustness compared to standard imitation learning baselines.

Introduction

Developing autonomous vehicles that are both safe and predictable remains a significant challenge. Current methodologies typically rely on either Reinforcement Learning (RL) or Imitation Learning (IL). RL often struggles with "reward misspecification," where agents exploit loopholes in hand-crafted rewards. Conversely, IL methods like Behavior Cloning suffer from "covariate shift," where small execution errors compound, leading the vehicle into unfamiliar and potentially catastrophic states.

Figure 1: Complex vehicle interactions in a real-world roundabout scenario from the Interaction Dataset.

While Generative Adversarial Imitation Learning (GAIL) addresses some IL limitations by allowing simulator-based practice, it typically operates on low-level actions (e.g., steering). In high-stakes environments like roundabouts, learning safety from scratch at this level is computationally difficult and prone to failure.

Methodology

SHAIL enhances the standard GAIL framework through two primary innovations:

  1. Hierarchical Structure: Instead of predicting raw control inputs, the high-level policy selects from a library of predefined "options" (e.g., specific target velocities). These options are executed by fixed, optimization-based low-level controllers designed for safety.
  2. Safety-Awareness Layer: A safety predictor estimates the probability ($p_{safe}$) of an option's safety. The high-level policy is reweighted to prioritize safe options. Furthermore, if an active option is predicted to become unsafe, control is immediately returned to the high-level policy for a new maneuver (early termination).

The authors reformulate the GAIL objective to match occupancy measures across this hierarchy, enabling the high-level policy to be trained using only low-level expert trajectories without requiring explicit high-level labels.

Experimental Setup

The framework was evaluated using the Interaction Dataset, which contains real-world motion tracks from complex urban scenarios. Testing was conducted in a custom OpenAI Gym-compatible simulator where the "ego" vehicle interacts with other agents following recorded real-world trajectories. The primary test scenario was the "DR_USA_Roundabout_FT" environment.

Results and Discussion

SHAIL was compared against several baselines, including Behavior Cloning (BC), standard GAIL, and an ablation model without the safety layer (HAIL).

Table 1: Quantitative performance comparison of SHAIL against baselines in in-distribution (top) and out-of-distribution (bottom) scenarios.

As shown in Table 1, SHAIL achieved the highest success rates in both in-distribution (77.7%) and out-of-distribution (64.4%) scenarios. The ablation study (HAIL) confirms that the safety-awareness layer is critical, contributing to a ~25% increase in success rate.

Key Insights

  • Safety as a Prerequisite: Treating safety as a filter for action selection effectively mitigates the reward misspecification issues common in RL.
  • Generalization: SHAIL demonstrated significantly better robustness to out-of-distribution data than raw end-to-end learning models.
  • The Jerkiness Trade-off: While safer, SHAIL exhibited higher acceleration Jensen-Shannon Divergence (JSD) compared to experts. This "jerky" behavior stems from the simplicity of the fixed-acceleration low-level controllers, highlighting a trade-off between safety and human-like comfort.

Conclusion and Future Work

SHAIL successfully integrates safety constraints into an imitation learning framework, providing a robust solution for complex urban navigation. Future research will focus on implementing more sophisticated low-level controllers to improve passenger comfort and exploring the combination of SHAIL with traditional RL rewards to further refine driving behavior beyond the limits of expert demonstrations.

Summary updated

BibTeX
@inproceedings{jamgochian2023shail,
  author = {Arec L. Jamgochian and Etienne Bührle and Johannes Fischer and Mykel J. Kochenderfer},
  title = {{SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10161449},
}
Figure