TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions

Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

Abstract

We present a new algorithm for predicting the near-term trajectories of road agents in dense traffic videos. Our approach is designed for heterogeneous traffic, where the road agents may correspond to buses, cars, scooters, bi-cycles, or pedestrians. We model the interactions between different road agents using a novel LSTM-CNN hybrid network for trajectory prediction. In particular, we take into account heterogeneous interactions that implicitly account for the varying shapes, dynamics, and behaviors of different road agents. In addition, we model horizon-based interactions which are used to implicitly model the driving behavior of each road agent. We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories. We outperform state-of-the-art methods on dense traffic datasets by 30%.

Introduction

Urban traffic environments, particularly in Asian cities, present a unique challenge for trajectory prediction due to their high density and extreme heterogeneity. Unlike standard highway datasets where agents (mostly cars) follow predictable lanes and behaviors, these environments feature a mix of buses, rickshaws, bicycles, and pedestrians sharing the same space. In such "organized chaos," agents cannot track every neighbor; instead, they prioritize interactions based on proximity and their immediate path. Existing models often fail here because they treat all neighbors equally or assume homogeneous agent dynamics.

Methodology: Weighted Interactions

TraPHic addresses these challenges using a hybrid architecture that combines Long Short-Term Memory (LSTM) networks for temporal modeling with Convolutional Neural Networks (CNNs) for spatial pattern recognition. The core innovation lies in its weighted interaction modules:

  1. Horizon-Based Interactions: Instead of a standard circular neighborhood, the model focuses on a semi-elliptical "horizon" ($H_i$) in front of the agent. This mimics human drivers who prioritize obstacles in their immediate forward path.
  2. Heterogeneous-Based Interactions: The model embeds agent-specific features—such as dimensions, velocity, and local density—into the state space. This allows the network to implicitly learn the physical constraints (e.g., turning radii) of different vehicle types from data.
Modeling horizon-based ($H_i$) and neighbor-based ($N_i$) interactions for an agent $a_i$.

The architecture generates "Horizon Maps" and "Neighbor Maps," which are processed through CNN layers to extract spatial features, then fused with the agent's historical state to predict future trajectories.

Experimental Results

The researchers evaluated TraPHic on three benchmarks: the new TRAF dataset (50 videos of dense Asian traffic), NGSIM (sparse highway data), and the Beijing Dataset (dense but less heterogeneous).

TraPHic demonstrated a significant performance leap, outperforming state-of-the-art methods like Social-GAN and CS-LSTM by 30% in Root Mean Square Error (RMSE) on the dense TRAF dataset. Ablation studies highlighted the importance of the horizon-based modeling, which contributed most significantly to the accuracy gains.

RMSE comparison of TraPHic variants against state-of-the-art CS-LSTM over a 5-second prediction horizon.

Key insights from the results include:

  • The "Horizon" Priority: Modeling the driver's field of view is more effective than a standard neighborhood, as drivers are "greedy" for the space immediately ahead.
  • Implicit Physics Learning: By observing agent dimensions and velocities, the model learns that a bus cannot maneuver through gaps accessible to a scooter.
  • Robustness: While optimized for dense chaos, TraPHic remains competitive on simpler, sparse datasets like NGSIM.

Conclusion and Future Work

TraPHic represents a major step toward navigating complex, real-world traffic. However, challenges remain in automating ground-truth trajectory labeling and moving from offline prediction to real-time integration within autonomous vehicle navigation stacks. Future work will also explore estimating agent dimensions directly from raw video input.

Summary updated

Abstract

We present a new algorithm for predicting the near-term trajectories of road agents in dense traffic videos. Our approach is designed for heterogeneous traffic, where the road agents may correspond to buses, cars, scooters, bi-cycles, or pedestrians. We model the interactions between different road agents using a novel LSTM-CNN hybrid network for trajectory prediction. In particular, we take into account heterogeneous interactions that implicitly account for the varying shapes, dynamics, and behaviors of different road agents. In addition, we model horizon-based interactions which are used to implicitly model the driving behavior of each road agent. We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories. We outperform state-of-the-art methods on dense traffic datasets by 30%.
BibTeX
@inproceedings{chandra2019traphic,
  author = {Rohan Chandra and Uttaran Bhattacharya and Aniket Bera and Dinesh Manocha},
  title = {{TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=EB6s2u08-sb},
}
Figure