Extracting Traffic Primitives Directly From Naturalistically Logged Data for Self-Driving Applications

Wenshuo Wang, Ding Zhao

TL;DR

This paper introduces a novel framework and a nonparametric Bayesian learning method (sticky HDP-HMM) to automatically extract "traffic primitives" – fundamental building blocks of driving scenarios – from real-world driving data. This allows for the generation of an infinite number of diverse and realistic traffic scenarios, crucial for training and evaluating self-driving systems.

What Problem Does This Paper Address?

Developing autonomous vehicles (AVs) capable of navigating complex and dynamic traffic environments requires them to understand and predict the behavior of other road users. While analyzing vast amounts of naturalistic driving data is key, manually extracting meaningful scenarios from this high-dimensional, time-series data is a significant bottleneck. Existing methods often rely on subjective manual extraction, which is time-consuming, resource-intensive, and may not cover the full spectrum of real-world driving situations. Furthermore, these manually defined scenarios are often inflexible for generating new, diverse test cases. This paper addresses the challenge of automatically learning these fundamental "traffic primitives" from raw data and using them to create a virtually limitless supply of new traffic scenarios for AV development.

Methods

The core of the paper's contribution lies in its proposed framework and a specific nonparametric Bayesian learning model. The framework aims to extract traffic primitives, model their relationships, and then use these to generate new traffic scenarios.

The key algorithmic contribution is the sticky Hierarchical Dirichlet Process Hidden Markov Model (sticky HDP-HMM). This model is designed to:

  • Handle Unknown Number of Primitives: Unlike traditional HMMs, the HDP component allows the model to infer the number of traffic primitives without prior specification.
  • Model Both Binary and Continuous Events: The model can simultaneously learn from discrete events (like a car entering or leaving a lane – "binary events") and continuous changes in vehicle states (like speed or acceleration – "continuous events").
  • Capture Temporal Dependencies: The HMM structure models the sequential nature of driving, where the current state depends on the previous one.
  • Promote Self-Transitions (Sticky Aspect): The "sticky" part of the HDP-HMM introduces a bias towards staying in the same primitive for a longer duration, which is a realistic characteristic of many driving behaviors.

The paper also details the emission model, which assumes observations are drawn from a Gaussian distribution, and the inference method, which uses Gibbs sampling.

Datasets

The paper utilizes data from the Safety Pilot Model Deployment (SPMD) database logged in Ann Arbor, Michigan. This database contains naturalistic driving data collected from instrumented vehicles. The data includes:

  • Ego vehicle states: speed, steering angle, acceleration/brake pedal position.
  • Surrounding vehicle information: relative distance, relative speed, and lateral displacement with respect to the lane boundary, captured by Mobileye sensors.

The data is recorded at 10 Hz. The paper does not introduce a new benchmark dataset but rather uses this existing, rich dataset to validate its proposed method.

Key Insights and Ideas

  • Decomposition into Primitives: The fundamental idea is that complex driving scenarios can be broken down into a finite set of "traffic primitives," analogous to how humans learn concepts.
  • Data-Driven Primitive Extraction: The nonparametric Bayesian approach allows for the discovery of these primitives directly from data, without human pre-definition, making it scalable to massive datasets.
  • Bridging Binary and Continuous Data: The ability of the sticky HDP-HMM to jointly model discrete (event-based) and continuous (state-based) traffic information is crucial for capturing the full richness of driving scenarios.
  • Scenario Generation Potential: By extracting primitives, the framework opens the door to generating an infinite variety of new traffic scenarios by combining and reshaping these learned building blocks, which is invaluable for AV testing.

Key Results

  • The proposed sticky HDP-HMM method successfully extracts traffic primitives from both binary (car appearance/disappearance) and continuous (vehicle dynamics) events in real-world driving data.
  • The method demonstrated its ability to detect step changes corresponding to car-following, lane-changing, and overtaking maneuvers.
  • The approach can automatically cluster primitives with similar attributes, assigning them to the same primitive ID.
  • Experiments on a day's worth of data from one driver resulted in the extraction of 22 kinds of primitives. For ten drivers, the method processed high-dimensional data (d=14 and d=17) and extracted primitives effectively.
  • The log-likelihood plots show convergence for both binary and continuous event learning, indicating the model's effectiveness.

Limitations and Future Work

  • Interpretability of Primitives: The authors acknowledge that while the extracted primitives are useful for scenario generation, they are not yet fully interpretable in a human-understandable way.
  • Human-Inspired Interpretability: Future work will focus on developing a human-inspired approach to automatically extract interpretable primitives, providing deeper insights into complex traffic scenarios.
  • Quantitative Evaluation of Primitive Levels: The paper notes that the distinct levels among extracted primitive clusters are not quantitatively clear. Future research will explore methods to comprehensively evaluate these distinct levels, potentially using Kullback-Leibler divergence or other clustering comparison techniques.

Summary updated

Abstract

Developing an automated vehicle, that can handle complicated driving scenarios and appropriately interact with other road users, requires the ability to semantically learn and understand driving environment, oftentimes, based on analyzing massive amounts of naturalistic driving data. An important paradigm that allows automated vehicles to both learn from human drivers and gain insights is understanding the principal compositions of the entire traffic, termed as traffic primitives. However, the exploding data growth presents a great challenge in extracting primitives from high-dimensional time-series traffic data with various types of road users engaged. Therefore, automatically extracting primitives is becoming one of the cost-efficient ways to help autonomous vehicles understand and predict the complex traffic scenarios. In addition, the extracted primitives from raw data should 1) be appropriate for automated driving applications and also 2) be easily used to generate new traffic scenarios. However, existing literature does not provide a method to automatically learn these primitives from large-scale traffic data. The contribution of this paper has two manifolds. The first one is that we proposed a new framework to generate new traffic scenarios from a handful of limited traffic data. The second one is that we introduce a nonparametric Bayesian learning method -- a sticky hierarchical Dirichlet process hidden Markov model -- to automatically extract primitives from multidimensional traffic data without prior knowledge of the primitive settings. The developed method is then validated using one day of naturalistic driving data. Experiment results show that the nonparametric Bayesian learning method is able to extract primitives from traffic scenarios where both the binary and continuous events coexist.
BibTeX
@article{wang2018extracting,
  author = {Wenshuo Wang and Ding Zhao},
  title = {{Extracting Traffic Primitives Directly From Naturalistically Logged Data for Self-Driving Applications}},
  journal = {IEEE RA-L},
  year = {2018},
  doi = {10.1109/lra.2018.2794604},
}
Figure