TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions

Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

Abstract

We present a new algorithm for predicting the near-term trajectories of road-agents in dense traffic videos. Our approach is designed for heterogeneous traffic, where the road-agents may correspond to buses, cars, scooters, bicycles, or pedestrians. We model the interactions between different road-agents using a novel LSTM-CNN hybrid network for trajectory prediction. In particular, we take into account heterogeneous interactions that implicitly accounts for the varying shapes, dynamics, and behaviors of different road agents. In addition, we model horizon-based interactions which are used to implicitly model the driving behavior of each road-agent. We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories. We outperform state-of-the-art methods on dense traffic datasets by 30%.

Introduction

Most existing trajectory prediction models are designed for structured, sparse environments such as highways or pedestrian walkways. These models often assume homogeneity (uniform agent types) and sparsity (ample maneuvering space). However, real-world urban traffic, particularly in Asian cities, is characterized by extreme density and heterogeneity. In these environments, a large bus must interact with agile motorcyclists, rickshaws, and pedestrians in a non-lane-based manner.

Standard models struggle in these scenarios because they often treat all nearby agents with equal importance. In a crowd of dozens, an agent primarily reacts to those in its immediate path. There is a critical need for models that can selectively attend to the most relevant interactions while accounting for the diverse physical constraints of different vehicle types. To address this, we introduce TraPHic and a new dataset, TRAF, containing 50 videos of highly dense traffic with 13 motorized vehicle types, pedestrians, and bicycles.

Methods: The TraPHic Architecture

The core of TraPHic is a hybrid network combining Long Short-Term Memory (LSTM) units with Convolutional Neural Networks (CNNs). The architecture focuses on two primary interaction types:

  1. Horizon-Based Interactions: The model defines a semi-elliptical "horizon" in front of the agent. Interactions within this zone are prioritized, mimicking human driving behavior that focuses on the immediate path forward.
  2. Heterogeneous-Based Interactions: The model implicitly learns how different agents (e.g., cars vs. bicycles) interact based on their size, velocity, and local traffic density (concentration).
The TraPHic hybrid LSTM-CNN architecture, showing the processing of ego, horizon, and neighbor state-spaces into interaction maps.

Individual LSTMs capture the temporal history of each agent. Their hidden states are pooled into "Horizon Maps" and "Neighbor Maps," which are then processed by CNNs to learn spatial relationships and local dependencies. The input state-space includes agent size, velocity, and a traffic concentration feature derived from fluid traffic flow theory.

Results and Discussion

TraPHic demonstrates a significant improvement over state-of-the-art methods, particularly in dense environments. On complex datasets, TraPHic improved the Root Mean Square Error (RMSE) by 0.78 meters, representing a 30% accuracy boost over prior methods like Social-GAN and CS-LSTM.

RMSE comparison over a 5-second prediction horizon. TraPHic (green) consistently maintains the lowest error compared to baselines and its own ablated versions.

Ablation studies confirm that both Horizon ($H_e$) and Heterogeneous ($H_o$) weighting are essential. Removing either component leads to significantly higher error rates. Furthermore, while optimized for dense traffic, TraPHic remains competitive on sparse datasets like NGSIM and KITTI, indicating robust generalization. Key insights include the importance of selective attention via the "horizon" concept and the effectiveness of implicit modeling through CNN-processed interaction maps.

Conclusion

TraPHic effectively navigates the complexities of dense, mixed traffic by prioritizing relevant interactions and accounting for agent diversity. While the current model relies on pre-extracted trajectory data, future work aims to develop an end-to-end system capable of predicting trajectories directly from raw video pixels. This would eliminate the labor-intensive step of manual tracking and labeling, moving closer to real-time application in autonomous systems.

Summary updated

Abstract

We present a new algorithm for predicting the near-term trajectories of road-agents in dense traffic videos. Our approach is designed for heterogeneous traffic, where the road-agents may correspond to buses, cars, scooters, bicycles, or pedestrians. We model the interactions between different road-agents using a novel LSTM-CNN hybrid network for trajectory prediction. In particular, we take into account heterogeneous interactions that implicitly accounts for the varying shapes, dynamics, and behaviors of different road agents. In addition, we model horizon-based interactions which are used to implicitly model the driving behavior of each road-agent. We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories. We outperform state-of-the-art methods on dense traffic datasets by 30%.
BibTeX
@inproceedings{chandra2019traphic,
  author = {Rohan Chandra and Uttaran Bhattacharya and Aniket Bera and Dinesh Manocha},
  title = {{TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=EB6s2u08-sb},
}
Figure