The Reasonable Crowd: Towards evidence-based and interpretable models of driving behavior

Bassam Helou, Aditya Dusi, Anne Collin, Noushin Mehdipour, Zhiliang Chen, Cristhian Lizarazo, Calin Belta, Tichakorn Wongpiromsarn, Radboud J. Duintjer Tebbens, Oscar BeijbomIROS 2021

Abstract

Autonomous vehicles must balance a complex set of objectives. There is no consensus on how they should do so, nor on a model for specifying a desired driving behavior. We created a dataset to help address some of these questions in a limited operating domain. The data consists of 92 traffic scenarios, with multiple ways of traversing each scenario. Multiple annotators expressed their preference between pairs of scenario traversals. We used the data to compare an instance of a rulebook, carefully hand-crafted independently of the dataset, with several interpretable machine learning models such as Bayesian networks, decision trees, and logistic regression trained on the dataset. To compare driving behavior, these models use scores indicating by how much different scenario traversals violate each of 14 driving rules. The rules are interpretable and designed by subject-matter experts. First, we found that these rules were enough for these models to achieve a high classification accuracy on the dataset. Second, we found that the rulebook provides high interpretability without excessively sacrificing performance. Third, the data pointed to possible improvements in the rulebook and the rules, and to potential new rules. Fourth, we explored the interpretability vs performance trade-off by also training non-interpretable models such as a random forest. Finally, we make the dataset publicly available to encourage a discussion from the wider community on behavior specification for AVs.

Introduction

How should an autonomous vehicle (AV) behave when objectives conflict? For example, should a car nudge closer to a pedestrian to avoid a potential rear-end collision, or vice versa? Currently, there is no industry-wide consensus on how to specify "reasonable" driving behavior. Most AV systems rely on complex cost functions that are difficult to tune and even harder to explain to regulators or the public. This research tackles the gap between evidence-based modeling—what humans actually prefer—and interpretability—explaining decisions through clear, prioritized rules.

Methodology: Rules and Rulebooks

The researchers bridge the gap between expert knowledge and data-driven learning using a three-pronged approach:

  1. Rule-Based Features: Instead of feeding raw sensor data into models, the system defines 14 interpretable driving rules (e.g., "Avoid collisions," "Stay in lane," "Maintain clearance"). Each driving path, or realization, is assigned a "violation score" for each rule.
  2. The Rulebook Approach: A "Rulebook" is a hierarchical structure (specifically a pre-order) that ranks rules by priority. For instance, avoiding a collision with a pedestrian is a higher priority than staying perfectly within lane markers.
  3. Machine Learning Comparison: Several models were trained to predict human preference based on rule violation scores. These ranged from interpretable models (Logistic Regression, Decision Trees, Bayesian Networks) to "black-box" models (Random Forests, Neural Networks).

The Reasonable Crowd Dataset

To ground these models in human judgment, the authors introduced the Reasonable Crowd dataset. It consists of 92 unique traffic scenarios set across two distinct simulated environments: an urban grid and a suburban residential area.

The study utilized two simulated environments: an urban grid (U) and a suburban neighborhood (S).

From these scenarios, 576 different realizations (traversals) were generated. The core of the dataset is 40,560 pairwise comparisons collected from 65 human annotators. Unlike datasets focused on human demonstrations, which are difficult to obtain for rare or dangerous events, this dataset focuses on human preferences between two specific outcomes. This allows for a clearer "ground truth" regarding social and ethical trade-offs in driving.

Annotators were presented with two different realizations (A and B) of the same traffic scenario and asked to choose which one a reasonable driver would prefer.

Key Results and Insights

  • Rules are Sufficient: The 14 expert-defined rules were enough for models to achieve high accuracy, suggesting that "end-to-end" deep learning may not be necessary to capture the nuances of reasonable driving.
  • The Interpretability "Tax" is Low: While the "black-box" Random Forest achieved the highest accuracy (82.3%), the interpretable Rulebook (augmented with a small decision tree) reached 78.4%. In safety-critical AV deployment, this minor performance hit is often a justifiable price for explainability.
  • Safety is Hard-Coded: A major advantage of the Rulebook is that it guarantees safety priorities. While a standard ML model might theoretically prioritize comfort over a minor collision due to training noise, the Rulebook prevents this by design.
  • Data as a Debugger: Human preference data identified where expert rules were too conservative (e.g., strict clearance thresholds) or where new rules were needed, such as yielding to pedestrians at crosswalks.

Conclusion and Future Work

While the current study is limited to sunny weather and static agents (where other vehicles do not react to the AV), it provides a framework for evidence-based behavior specification. Future work will focus on scaling the Rulebook to more complex environments and using combinatoric methods to automatically optimize rule priority structures from human data. The dataset is publicly available at github.com/bassam-motional/Reasonable-Crowd to encourage community-wide discussion on AV behavior.

Summary updated

Abstract

Autonomous vehicles must balance a complex set of objectives. There is no consensus on how they should do so, nor on a model for specifying a desired driving behavior. We created a dataset to help address some of these questions in a limited operating domain. The data consists of 92 traffic scenarios, with multiple ways of traversing each scenario. Multiple annotators expressed their preference between pairs of scenario traversals. We used the data to compare an instance of a rulebook, carefully hand-crafted independently of the dataset, with several interpretable machine learning models such as Bayesian networks, decision trees, and logistic regression trained on the dataset. To compare driving behavior, these models use scores indicating by how much different scenario traversals violate each of 14 driving rules. The rules are interpretable and designed by subject-matter experts. First, we found that these rules were enough for these models to achieve a high classification accuracy on the dataset. Second, we found that the rulebook provides high interpretability without excessively sacrificing performance. Third, the data pointed to possible improvements in the rulebook and the rules, and to potential new rules. Fourth, we explored the interpretability vs performance trade-off by also training non-interpretable models such as a random forest. Finally, we make the dataset publicly available to encourage a discussion from the wider community on behavior specification for AVs. Please find it at github.com/bassam-motional/Reasonable-Crowd.
BibTeX
@inproceedings{helou2021reasonable,
  author = {Bassam Helou and Aditya Dusi and Anne Collin and Noushin Mehdipour and Zhiliang Chen and Cristhian Lizarazo and Calin Belta and Tichakorn Wongpiromsarn and Radboud J. Duintjer Tebbens and Oscar Beijbom},
  title = {{The Reasonable Crowd: Towards evidence-based and interpretable models of driving behavior}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
}
Figure