Introduction
Pedestrian trajectory prediction (PTP) is a cornerstone of intelligent surveillance and autonomous navigation. However, traditional research predominantly focuses on single-camera "aerial" views, which fail to capture the complexity of modern infrastructure like airports or transit hubs where pedestrians move through interconnected camera networks. This paper introduces a novel framework, HM³GAT, designed to predict trajectories across multi-camera environments by jointly modeling spatial paths and camera-to-camera transitions.
The Multi-Camera Trajectory Prediction Problem
Existing Multi-Camera Trajectory Prediction (MCTP) approaches often treat destination prediction as a simple classification task. This paper argues that trajectory (the "how") and destination (the "where") are intrinsically coupled. A pedestrian's current walking pattern provides clues about their target camera, while their intended destination constrains their likely path. HM³GAT addresses this by predicting a "latent goal"—the specific departure point in the current scene—to guide the trajectory decoder.
Methodology: The HM³GAT Framework
The Hierarchical Multi-Supervision Multi-Interaction Graph Attention Network (HM³GAT) architecture consists of three primary components designed to capture the nuances of multi-camera movement.
1. Hierarchical Social Interaction (HMGAT)
Social forces are modeled at two levels:
- Person Level (H1): A Multi-head Graph Attention Network (GAT) models interactions within a single camera view, considering visibility, distance, and relative displacement between individuals.
- Group Level (H2): This level captures how groups of people interact across different camera scenes, acknowledging that pedestrians often move in flows or clusters.
2. Latent Goal Decoder (HMC)
This module performs Hierarchical Multi-label Classification to predict the pedestrian's "latent goal." Rather than just identifying the next camera, it predicts the specific "departure" (exit/entrance point) the pedestrian is targeting. This prediction is masked by the known camera network topology to ensure physically possible transitions.
3. Information Fusion and Trajectory Decoding
A Transformer-based encoder fuses historical motion data, social interaction features, and the predicted latent goals. A specialized Dual-stream LSTM (DLSTM) then generates the future spatial coordinates. The "Multi-Supervision" aspect refers to the joint optimization of trajectory displacement error and destination classification accuracy.
The mcScenes Dataset
To evaluate the model, the authors introduced mcScenes, a new dataset collected from a laboratory environment equipped with four cameras and eleven transition points ("departures"). Unlike existing datasets, mcScenes includes detailed manual annotations for interaction categories such as "leader-follower" and "collision avoidance," providing a richer ground truth for complex social modeling.
Key Results and Insights
HM³GAT demonstrated superior performance over adapted single-camera baselines (e.g., Social-LSTM, STGAT) on the mcScenes dataset:
- Spatial Accuracy: The model achieved a minimum Average Displacement Error (minADE) of 0.049 and a minimum Final Displacement Error (minFDE) of 0.095.
- Destination Accuracy: The next-camera prediction reached an accuracy of 78.9%.
- Ablation Findings: Removing the Latent Goal Decoder significantly increased trajectory errors, confirming that destination-awareness is a critical prior for accurate path prediction.
Conclusion and Future Work
HM³GAT represents a significant step toward holistic multi-camera surveillance. By integrating topological constraints and hierarchical social interactions, the model produces more realistic and accurate trajectories. While the model handles standard social behaviors well, future work will focus on improving performance during high-tension collision avoidance scenarios and scaling the framework to larger, more complex environments like multi-story transit hubs.
Summary updated