Predicting the behavior of interacting humans by fusing data from multiple sources

Erik J. Schlicht, Ritchie Lee, David H. Wolpert, Mykel J. Kochenderfer, Brendan D. Tracey

Introduction

Designing safe systems for high-stakes environments like aviation or medicine requires accurate predictions of human behavior. Traditionally, this necessitates Human-In-The-Loop (HITL) simulations. While these "high-fidelity" (Hi-Fi) simulations provide high-quality data from expert participants using realistic hardware, they are prohibitively expensive and time-consuming. Conversely, "low-fidelity" (Lo-Fi) data—such as that collected via crowdsourcing platforms like Amazon Mechanical Turk—is inexpensive and abundant but often lacks the accuracy of expert behavior in real-world scenarios.

This paper, originally published in 2012, addresses this "fidelity gap" by proposing a multi-fidelity framework to fuse these disparate data sources into highly accurate predictive models, even when expert data is scarce.

The Visual Self-Separation Testbed

To evaluate their framework, the authors utilized a visual self-separation game as a testbed. In this scenario, two pilots find themselves on a near-collision course. Each must decide on a heading change to avoid a crash while simultaneously attempting to minimize deviation from their original flight path.

The visual self-separation game involves two aircraft (s1, s2) making decisions (a1, a2) to avoid collision while reaching target states (sf1, sf2).

The study compared two distinct levels of fidelity:

  • High-Fidelity (Hi-Fi): A simulation featuring a full instrument panel and "expert" behavior (simulated via specific utility weights).
  • Low-Fidelity (Lo-Fi): A simplified version where the instrument panel is removed, forcing participants to rely solely on out-the-window visual cues.

Methodology: Model-Free vs. Model-Based Fusion

The core of the research lies in how these data sources are combined. The authors explored two primary approaches:

1. Model-Free Approaches

These methods utilize Locally Weighted (LW) Regression. In the multi-fidelity variant, Lo-Fi data is used to train a "pre-predictor." The output of this pre-predictor then serves as an additional feature for the model trained on the limited Hi-Fi data.

2. Model-Based Approaches

These methods incorporate a structural model of human decision-making, specifically assuming a level-$k$ reasoning strategy. In this context, a level-1 ($k=1$) strategy assumes the other pilot will maintain their current course, and the actor reacts accordingly. The model estimates "utility weights" that represent the trade-off between collision avoidance and path efficiency.

  • MAP (Maximum A Posteriori) Multi-fidelity: This approach finds the optimal utility weights by combining the likelihood of observed actions from both Lo-Fi and Hi-Fi sources.
  • Bayesian Multi-fidelity: This more robust method accounts for uncertainty by sampling across a distribution of utility weights, weighted by their ability to explain both data sets.

Key Results and Insights

The experimental results highlight a significant advantage for model-based methods. By assuming humans are "bounded rational" agents, the algorithms could learn the underlying logic of the behavior from Lo-Fi data, even when specific actions differed from the Hi-Fi experts.

Average predictive efficiency across different fusion methods. Model-based multi-fidelity approaches (red and green) significantly outperform model-free methods (blue), especially when expert Hi-Fi data is scarce.

Key findings include:

  • Predictive Efficiency: Model-based multi-fidelity methods achieved high efficiency (near 0.9) with as few as 10 to 20 Hi-Fi samples when supplemented by 1,000 Lo-Fi samples.
  • The Failure of Model-Free Methods: Standard regression-based fusion performed poorly, often failing to reach 0.7 efficiency. This suggests that raw data fusion is insufficient without an underlying model of human intent.
  • Sufficiency of Level-1 Reasoning: Consistent with previous behavioral research, the authors found that assuming a shallow depth of reasoning ($k=1$) was sufficient to model interactions in these stressful, time-critical scenarios.

Conclusion

The research demonstrates that incorporating a model of human decision-making is crucial for bridging the fidelity gap. Lo-Fi data serves as an effective "prior," narrowing the search space for human-like behavior, which sparse Hi-Fi data can then refine. This multi-fidelity approach offers a path toward developing highly accurate predictive models for complex human-machine systems without the prohibitive cost of large-scale expert simulations.

Summary updated

BibTeX
@inproceedings{schlicht2012predicting,
  author = {Erik J. Schlicht and Ritchie Lee and David H. Wolpert and Mykel J. Kochenderfer and Brendan D. Tracey},
  title = {{Predicting the behavior of interacting humans by fusing data from multiple sources}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2012},
}
Figure