Pedestrian Trajectory Prediction Using Dynamics-based Deep Learning

Honghui Wang, Weiming Zhi, Gustavo Batista, Rohitash Chandra

Abstract

This paper introduces a "dynamics-based deep learning" (DDL) framework that integrates mathematically guaranteed stable dynamical systems into Transformer-based models for pedestrian trajectory prediction. By forcing predicted paths to converge toward a goal while learning social navigation behaviors, the model achieves a significant performance boost—outperforming state-of-the-art models by over 23% on standard benchmarks.

Introduction: The Challenge of Human Movement Prediction

Predicting pedestrian trajectories is a critical task for autonomous vehicles and social robots. Current state-of-the-art models, primarily based on Recurrent Neural Networks (RNNs) and Transformers, face two significant hurdles. First, they often function as "black boxes," providing little insight into the reasoning behind a specific predicted path. Second, they frequently lack physical constraints; without an explicit understanding that human movement is goal-directed, these models can produce physically unrealistic trajectories that fail to converge on logical destinations.

The proposed Dynamics-based Deep Learning (DDL) framework seeks to bridge the gap between rigid, explainable physics-based models and flexible, but opaque, deep learning architectures.

Methodology: The DDL Framework

The DDL framework integrates Lyapunov stability theory directly into a neural network architecture to ensure that predicted trajectories are both socially aware and mathematically guaranteed to reach a destination.

The Dynamics-based Deep Learning (DDL) framework architecture, illustrating the integration of goal estimation and stable dynamical systems.

The framework operates in three primary stages:

  • Goal Estimation: The model analyzes a pedestrian's historical path using a $\gamma$-soft Dynamic Time Warping (DTW) algorithm and K-means clustering. By comparing current movement to an "expert repository" of historical trajectories, it estimates the most likely equilibrium point or destination.
  • Asymptotically Stable Dynamical System (DS): Rather than predicting raw 2D coordinates, the system defines a dynamical system governed by a stable attractor. This mathematical constraint ensures that the trajectory will always converge to the estimated goal.
  • Transformer-based Learning (STAR): A Spatio-Temporal grAph tRansformer (STAR) is employed to learn the parameters of a positive-definite matrix ($P$). This matrix "shapes" the dynamical system, allowing the model to incorporate complex social behaviors—such as obstacle avoidance and maintaining social distance—while the underlying math maintains stability. To ensure $P$ remains positive-definite during training, the authors utilize an LU decomposition strategy.

Experimental Evaluation

The DDL framework was evaluated using the ETH and UCY datasets, the industry standards for pedestrian trajectory prediction. These datasets consist of real-world video footage from diverse environments, such as university campuses and busy sidewalks, translated into 2D coordinate trajectories.

Results and Insights

The integration of stability as a prior significantly improves prediction accuracy and interpretability.

Quantitative Performance

DDL demonstrated superior performance across five benchmark datasets, showing an average improvement of 23.1% in Average Displacement Error (ADE) and 26.4% in Final Displacement Error (FDE) compared to the baseline STAR model. It also outperformed other prominent models, including PECNet, Goal-GAN, and Social-STGCNN.

Qualitative Success and Explainability

Because the model predicts velocity based on a gradient descent toward a goal, the resulting paths are inherently more interpretable. Visualizations confirm that the model successfully learns implicit social norms.

Qualitative results showing the model predicting social navigation and collision avoidance behaviors in a crowded environment.

As shown in the qualitative analysis, the model can predict complex maneuvers, such as a pedestrian veering or slowing down to avoid a collision, without requiring hand-coded "social force" rules. The Transformer effectively adjusts the "shaping matrix" $P$ to account for the surrounding environment.

Conclusion and Future Work

The DDL framework represents a significant step toward more reliable and explainable trajectory prediction. While currently limited by the use of positive-definite matrices—which can restrict the sharpness of predicted turns—the authors propose future work involving "diffeomorphic transforms" to enable more complex curvatures while maintaining stability. Further testing in extremely congested social settings will also be necessary to evaluate the robustness of the learned shaping matrices.

Summary updated

BibTeX
@inproceedings{wang2024pedestrian,
  author = {Honghui Wang and Weiming Zhi and Gustavo Batista and Rohitash Chandra},
  title = {{Pedestrian Trajectory Prediction Using Dynamics-based Deep Learning}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  url = {https://openreview.net/forum?id=NCQqai9FdQ},
}
Figure