Interpretable Social Anchors for Human Trajectory Forecasting in Crowds

Parth Kothari, Brian Sifringer, Alexandre Alahi

Abstract

Human trajectory forecasting in crowds, at its core, is a sequence prediction problem with specific challenges of capturing inter-sequence dependencies (social interactions) and consequently predicting socially-compliant multimodal distributions. In recent years, neural network-based methods have been shown to outperform hand-crafted methods on distance-based metrics. However, these data-driven methods still suffer from one crucial limitation: lack of interpretability. To overcome this limitation, we leverage the power of discrete choice models to learn interpretable rule-based intents, and subsequently utilise the expressibility of neural networks to model scene-specific residual. Extensive experimentation on the interaction-centric benchmark TrajNet++ demonstrates the effectiveness of our proposed architecture to explain its predictions without compromising the accuracy.

Introduction

Predicting human movement in crowded environments is a "socially-compliant multimodal" challenge. Pedestrians do not merely follow linear paths; they dynamically adjust their trajectories to avoid collisions, follow leaders, and respect personal space. Historically, researchers relied on knowledge-based models like Social Forces or Discrete Choice Models (DCMs), which offered high interpretability but struggled with long-term dependencies and complex non-linear interactions.

In contrast, modern deep learning approaches using Neural Networks (NNs) have achieved state-of-the-art accuracy but often operate as "black boxes." This lack of transparency is a significant barrier for safety-critical applications, such as autonomous navigation. This paper introduces SAnchor, a hybrid architecture that bridges this gap by disentangling human motion into high-level discrete intents ("social anchors") and low-level scene-specific refinements.

Methodology

The SAnchor framework operates in three distinct stages to ensure both accuracy and interpretability:

  1. Social Anchors (Discretization): Instead of directly predicting continuous coordinates, the model considers a discrete set of potential future intents. These are represented as a radial grid of "anchors" normalized to the pedestrian's current speed and direction.
  2. Discrete Choice Modeling (DCM): The model assigns probabilities to these anchors using a hybrid utility function. This function integrates hand-crafted social rules—such as "keep direction," "avoid occupancy," "leader-follower," and "collision avoidance"—with learned features from an LSTM-based encoder that captures long-term motion history.
  3. Residual Refinement: To achieve smooth, continuous movement, a neural network decoder predicts a "scene-specific residual" (modeled as a bivariate Gaussian) for each anchor. This refinement shifts the discrete anchor to a precise coordinate.
The SAnchor architecture combines a neural network encoder-decoder with a Discrete Choice Model (DCM) to predict anchor probabilities and scene-specific residuals.

A critical technical component is Direction Normalization. By rotating the input scene at every time step so the pedestrian of interest always faces "forward," the model becomes rotation-invariant. This allows the system to focus purely on the geometry of social interactions rather than absolute map coordinates.

Results and Interpretability

The authors evaluated SAnchor on TrajNet++, a benchmark specifically designed for interaction-centric trajectory forecasting. The model demonstrated superior performance in both accuracy and safety metrics. On synthetic data, SAnchor achieved an Average Displacement Error (ADE) of 0.22m, outperforming SGAN (0.27m) and CVAE (0.26m). More importantly, it significantly reduced "prediction collisions," maintaining a collision rate of only 0.4% on synthetic data and the lowest rate (4.2%) on real-world data among tested models.

The primary advantage of SAnchor is its inherent interpretability. By forcing the model to select from discrete anchors, the system can generate "activation maps" that visualize the influence of specific behavioral rules.

Visualization of activation maps showing how different behavioral rules (DCM components) influence the final predicted path.

As shown in the qualitative "Final Maps," one can observe how different rules—such as leader-follower or collision avoidance—compete or cooperate to produce the final predicted path. This provides a high-level rationale for the model's decisions that standard neural networks cannot offer.

Conclusion

SAnchor successfully integrates the "why" of rule-based models with the "how well" of deep learning. By leveraging discrete social anchors and DCMs, the model provides transparent, socially-compliant trajectory predictions without sacrificing accuracy. While the current implementation relies on a predefined set of social rules, future research could explore the automatic discovery of new rules from data, further enhancing the flexibility and explanatory power of the system.

Summary updated

Abstract

Human trajectory forecasting in crowds, at its core, is a sequence prediction problem with specific challenges of capturing inter-sequence dependencies (social interactions) and consequently predicting socially-compliant multimodal distributions. In recent years, neural network-based methods have been shown to outperform hand-crafted methods on distance-based metrics. However, these data-driven methods still suffer from one crucial limitation: lack of interpretability. To overcome this limitation, we leverage the power of discrete choice models to learn interpretable rule-based intents, and subsequently utilise the expressibility of neural networks to model scene-specific residual. Extensive experimentation on the interaction-centric benchmark TrajNet++ demonstrates the effectiveness of our proposed architecture to explain its predictions without compromising the accuracy.
BibTeX
@inproceedings{kothari2021interpretable,
  author = {Parth Kothari and Brian Sifringer and Alexandre Alahi},
  title = {{Interpretable Social Anchors for Human Trajectory Forecasting in Crowds}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  url = {https://openreview.net/forum?id=uAXl8IOj9R},
}

Loading related content...

Figure