StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology

Yanliang Zhu, Deheng Qian, Dongchun Ren, Huaxia Xia

TL;DR

This paper introduces StarNet, a novel deep learning architecture for pedestrian trajectory prediction that models collective pedestrian interactions more effectively than previous methods. By employing a unique "star" topology with a central hub network and multiple host networks, StarNet achieves higher accuracy and significantly improved computational efficiency, making it more practical for real-world applications.

What Problem Does This Paper Address?

Predicting where pedestrians will move next is crucial for applications like autonomous driving and robotics, enabling safer and smoother navigation. However, this task is inherently challenging due to the complex and dynamic interactions between pedestrians. Existing methods often simplify these interactions by focusing only on pairwise relationships, which can lead to inaccurate predictions, especially in crowded scenes. Furthermore, modeling all pairwise interactions becomes computationally prohibitive as the number of pedestrians increases. This paper aims to address both the accuracy and efficiency limitations of current pedestrian trajectory prediction models by proposing a method that captures collective interactions.

Methods

StarNet introduces a novel "star" topology for its deep neural network architecture. This topology consists of two main components:

  • Hub Network: This central component takes the observed trajectories of all pedestrians as input. It processes this collective information to generate a comprehensive spatio-temporal representation of the crowd's interactions. This representation is designed to be invariant to translation and rotation, ensuring robustness.
  • Host Networks: For each individual pedestrian, there is a dedicated host network. Each host network receives the collective interaction representation from the hub network, along with the specific pedestrian's observed trajectory. It then uses this combined information to predict the pedestrian's future trajectory.

The key innovation lies in how interactions are modeled. Instead of pairwise calculations, the hub network aggregates information from all pedestrians, allowing it to capture "collective influence." This representation is then shared among all host networks. This approach is computationally efficient because the hub network's processing is linear with respect to the number of pedestrians, and all host networks can share weights, further reducing computational load and improving generalization.

Datasets

The paper evaluates StarNet on two widely used human crowded trajectory datasets:

  • ETH [24]
  • UCY [25]

These datasets comprise 5 scenes each, featuring challenging scenarios like collision avoidance and changing directions. The authors follow a standard practice of training on 4 scenes and testing on the remaining one.

Key Insights and Ideas

  • Collective Interactions Matter: The core insight is that modeling pedestrian behavior solely through pairwise interactions is insufficient. Capturing the "collective influence" of all pedestrians in a scene leads to more accurate predictions.
  • Star Topology for Efficiency and Accuracy: The proposed star topology elegantly separates the global interaction modeling (hub network) from individual prediction (host networks). This design is key to achieving both improved accuracy and computational efficiency.
  • Shared Weights for Generalization: Using shared weights across all host networks allows the model to learn generalizable pedestrian behavioral patterns, making it robust to different scenarios and reducing the number of parameters.
  • Translation and Rotation Invariance: Explicitly handling translation invariance in the spatial representation and using random rotation during training ensures the model is robust to the absolute position and orientation of the scene.

Key Results

  • Accuracy: StarNet outperforms state-of-the-art methods like Social LSTM, Social GAN, and Social Attention in terms of Average Displacement Error (ADE) and Final Displacement Error (FDE) on the ETH and UCY datasets. For instance, StarNet achieves an Average ADE of 0.30 and an Average FDE of 0.57, significantly better than the closest competitors.
  • Robustness: StarNet exhibits smaller variances in ADE and FDE across different scenes, indicating its robustness to scene variations.
  • Computational Efficiency: StarNet is remarkably faster than other advanced methods. It is 51 times faster than Social Attention, 7 times faster than Social LSTM, and 3 times faster than Social GAN. Its inference time is reported as 0.073 seconds per scene.
  • Parameter Efficiency: StarNet also uses significantly fewer parameters (31.90 Kilo) compared to other methods like Social GAN (874.95 Kilo).

Limitations and Future Work

  • Failure Cases: While generally accurate, the paper acknowledges that StarNet can still exhibit failure cases, particularly in scenarios with highly multimodal future trajectories. The authors suggest that these failures might be due to the inherent ambiguity of human behavior.
  • Plausible but Incorrect Predictions: In some instances, predicted trajectories might be plausible but do not match the ground truth. The authors argue that this is acceptable as long as the predictions are safe and avoid collisions.
  • Future Work: The paper does not explicitly detail future work directions, but the success in capturing collective interactions and the efficiency gains suggest potential for application in more complex multi-agent systems and real-time robotic navigation. Further exploration of handling multimodal predictions more effectively could also be a promising avenue.

Summary updated

Abstract

Pedestrian trajectory prediction is crucial for many important applications. This problem is a great challenge because of complicated interactions among pedestrians. Previous methods model only the pairwise interactions between pedestrians, which not only oversimplifies the interactions among pedestrians but also is computationally inefficient. In this paper, we propose a novel model StarNet to deal with these issues. StarNet has a star topology which includes a unique hub network and multiple host networks. The hub network takes observed trajectories of all pedestrians to produce a comprehensive description of the interpersonal interactions. Then the host networks, each of which corresponds to one pedestrian, consult the description and predict future trajectories. The star topology gives StarNet two advantages over conventional models. First, StarNet is able to consider the collective influence among all pedestrians in the hub network, making more accurate predictions. Second, StarNet is computationally efficient since the number of host network is linear to the number of pedestrians. Experiments on multiple public datasets demonstrate that StarNet outperforms multiple state-of-the-arts by a large margin in terms of both accuracy and efficiency.
BibTeX
@inproceedings{zhu2019starnet,
  author = {Yanliang Zhu and Deheng Qian and Dongchun Ren and Huaxia Xia},
  title = {{StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=GA0HPLwEAN},
}
Figure