Abstract
Pedestrian trajectory prediction requires modeling both individual multimodality and complex social interactions. Traditional state-of-the-art models often rely on computationally expensive post-processing steps, such as K-means clustering, to generate final predictions. We present TUTR (Trajectory Unified TRansformer), an architecture that unifies these components into a single Transformer encoder-decoder framework. TUTR eliminates the need for non-parallel post-processing, achieving state-of-the-art accuracy while operating 10x to 40x faster than previous top-tier methods.
Introduction
Predicting pedestrian movement is a fundamental challenge for autonomous systems, primarily due to the inherent multimodality of human intent and the influence of social interactions. Existing models typically generate a large set of potential trajectories and then apply clustering techniques to select a representative subset. However, this approach introduces significant latency, particularly in crowded environments, and often fails to provide reliable probability estimates for each path. TUTR addresses these limitations by treating trajectory prediction as a set-prediction problem, similar to modern object detection frameworks.
Methodology: The TUTR Architecture
TUTR utilizes a Transformer-based architecture to process trajectories through four distinct stages, bypassing traditional sampling-and-clustering bottlenecks.
- Global Prediction: The model identifies common "motion modes" (e.g., straight, left turn) from training data. These modes serve as initial tokens for the Transformer.
- Mode-Level Encoder: This stage models the relationships between different motion modes to capture the variety of possible human movements.
- Social-Level Decoder: The decoder incorporates "neighbor embeddings" to account for nearby pedestrians, ensuring that predicted paths are socially acceptable and collision-free.
- Dual Prediction Head: TUTR outputs both diverse trajectories and their associated probabilities in a single parallel step. This design enables the model to optimize for Brier metrics, which reward accurate probability calibration.
Experimental Evaluation
TUTR was evaluated on standard benchmarks, including the ETH/UCY datasets and the Stanford Drone Dataset (SDD). These datasets provide a range of real-world scenarios, from simple bird's-eye views to complex campus environments with high pedestrian density.
Results and Discussion
TUTR demonstrates a superior balance between accuracy and computational efficiency.
Accuracy and Probability Calibration
TUTR achieved state-of-the-art results on Brier-ADE and Brier-FDE metrics. Ablation studies confirmed that the "Global Prediction" stage is critical; replacing pre-calculated motion modes with standard learnable vectors resulted in a significant degradation of performance (e.g., ADE increased from 7.76 to 17.26 on SDD).
Computational Efficiency
A key advantage of TUTR is its scalability. In dense scenes with over 80 pedestrians, TUTR is approximately 40x faster than models like MemoNet and SocialVAE. By shifting the diversity logic into the parallelizable Transformer architecture, TUTR removes the CPU-bound bottleneck of clustering algorithms.
Conclusion
TUTR represents a significant step forward in real-time trajectory prediction by unifying multimodal and social modeling into a single, efficient Transformer framework. While highly effective, future iterations could benefit from incorporating environmental map data to further refine predictions in spatially constrained areas.
Summary updated