Meta-path Analysis on Spatio-Temporal Graphs for Pedestrian Trajectory Prediction

Aamir Hasan, Pranav Sriram, Katherine Driggs CampbellICRA 2022 · DOI · OpenReview

Abstract

Predicting pedestrian trajectories in dense, crowded environments remains a significant challenge for autonomous systems. Traditional Spatio-Temporal (ST) graph models often fail to capture the long-term contextual relationships between individuals. This paper presents MESRNN (Meta-path Enhanced Structural Recurrent Neural Network), a novel framework that utilizes "meta-paths"—structured walks through a graph—to represent complex spatial and temporal interactions. By incorporating these higher-order relationships, MESRNN achieves a 32% improvement in prediction accuracy over existing state-of-the-art methods.

Introduction

Accurate prediction of human movement is essential for safe and efficient autonomous navigation. Humans navigate based on a combination of personal goals, environmental obstacles, and the movements of others. While Spatio-Temporal (ST) graphs are a standard tool for modeling these interactions, current approaches typically focus on instantaneous spatial proximity. This limitation often results in "socially unaware" predictions that fail to account for group dynamics or historical context.

Spatio-temporal graph representation of pedestrian movement across multiple time steps.

Methods: The MESRNN Framework

The core of the MESRNN approach is the integration of meta-path analysis into a Structural RNN architecture.

The ST-Graph Representation

The scene is modeled as a graph where nodes represent pedestrian positions. Spatial edges connect individuals at the same time step, while temporal edges track a single individual's movement across consecutive time steps.

Meta-path Innovation

To capture deeper context, the authors define four types of meta-paths (paths of length 2):

  • Spatial-Spatial (SS): Captures indirect relationships between pedestrians through a common neighbor.
  • Spatial-Temporal (ST): Relates a person's current position to a neighbor's previous position.
  • Temporal-Spatial (TS): Relates a person's previous position to a neighbor's current position.
  • Temporal-Temporal (TT): Models an individual's own movement history over two time steps.
Factor graph representation showing how node and meta-path factors are integrated into the model.

These meta-path features are processed by EdgeRNNs to condense information, which is then combined with the pedestrian's current state in a NodeRNN to predict future offsets. This architecture is inherently scalable, as parameters are shared across similar edge types regardless of the number of pedestrians.

Experimental Evaluation

The model was rigorously tested on the ETH (ETH-UNIV, ETH-HOTEL) and UCY (ZARA01, ZARA02, UCY-UNIV) datasets. These datasets provide real-world scenarios with varying crowd densities and complex social interactions, serving as the benchmark for trajectory prediction.

Key Insights and Results

  • Contextual History: By explicitly modeling "where I am relative to where you were," MESRNN captures the underlying intent and flow of a crowd more effectively than models relying solely on current distance.
  • Quantitative Performance: MESRNN demonstrated a 32% average improvement in Average Displacement Error (ADE) compared to the standard Structural-RNN (SRNN).
  • Social Compliance: Qualitative analysis shows that MESRNN better maintains intended direction and pace in dense crowds, avoiding the "drift" common in baseline models. The performance gains were most pronounced in the complex UCY datasets.

Limitations and Future Directions

While MESRNN offers significant improvements, it currently produces unimodal predictions. Future work aims to extend the model to predict a probability distribution of multiple potential paths to account for human uncertainty. Additionally, while the system is computationally efficient, further optimization through "intelligent selection" of semantically valuable meta-paths is being explored for real-time integration into robotic navigation stacks.

Summary updated

Abstract

Spatio-temporal graphs (ST-graphs) have been used to model time series tasks such as traffic forecasting, human motion modeling, and action recognition. The high-level structure and corresponding features from ST-graphs have led to improved performance over traditional architectures. However, current methods tend to be limited by simple features, despite the rich information provided by the full graph structure, which leads to inefficiencies and suboptimal performance in downstream tasks. We propose the use of features derived from meta-paths, walks across different types of edges, in ST-graphs to improve the performance of Structural Recurrent Neural Network. In this paper, we present the Meta-path Enhanced Structural Recurrent Neural Network (MESRNN), a generic framework that can be applied to any spatio-temporal task in a simple and scalable manner. We employ MESRNN for pedestrian trajectory prediction, utilizing these meta-path based features to capture the relationships between the trajectories of pedestrians at different points in time and space. We compare our MESRNN against state-of-the-art ST-graph methods on standard datasets to show the performance boost provided by meta-path information. The proposed model consistently outperforms the baselines in trajectory prediction over long time horizons by over 32\%, and produces more socially compliant trajectories in dense crowds. For more information please refer to the project website at https://sites.google.com/illinois.edu/mesrnn/home.
BibTeX
@misc{hasan2022meta,
  author = {Aamir Hasan and Pranav Sriram and Katherine Driggs Campbell},
  title = {{Meta-path Analysis on Spatio-Temporal Graphs for Pedestrian Trajectory Prediction}},
  year = {2022},
  eprint = {2202.13427},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2202.13427},
}
Figure