DifTraj: Diffusion Inspired by Intrinsic Intention and Extrinsic Interaction for Multi-Modal Trajectory Prediction

Yanghong Liu, Xingping Dong, Yutian Lin, Mang Ye

Introduction

Predicting human movement is a fundamental challenge for autonomous systems, such as self-driving cars and social robots. The difficulty lies in the stochastic and multi-modal nature of human behavior: given a specific past trajectory, there are numerous plausible future paths an individual might take.

Existing research has largely focused on extrinsic interactions, such as avoiding collisions with other pedestrians. However, these models often overlook intrinsic intentions—the latent goals or destinations that drive a person's movement. Without accounting for the "why" behind a path, predicted trajectories can drift or appear aimless, even if they are socially acceptable. DifTraj is a novel diffusion-based framework designed to bridge this gap by explicitly modeling both internal goals and external social factors.

Methodology

DifTraj is an end-to-end framework built upon three core pillars designed to capture the complexity of human navigation:

  1. Interaction Feature Extraction: To understand social dynamics, the model employs a combination of Long Short-Term Memory (LSTM) networks and Graph Attention Networks (GAT). This module captures how pedestrians influence one another across space and time. A specialized "time-attention" layer weighs historical timestamps to prioritize the most relevant past movements.
  2. Intention Modeling: This module introduces a core innovation by using a Transformer-based encoder to predict a pedestrian's likely destination. By treating the destination as a proxy for latent intention, the model provides a stable "anchor" for the trajectory, preventing the common issue of path divergence.
  3. Diffusion-based Trajectory Inference: Unlike standard diffusion models that begin with pure Gaussian noise, DifTraj utilizes the extracted intention and interaction features to estimate a coarse initial distribution. This "leapfrog initialization" allows the diffusion denoising process to refine paths into precise, socially-compliant trajectories more efficiently.
The DifTraj architecture, integrating interaction feature extraction, intention modeling, and a diffusion-based denoising module.

To further stabilize training and ensure high-quality outputs, the authors introduced a Sample Consistency Loss ($L_{sc}$). This loss function constrains the variance between multiple generated samples, forcing them to remain consistent with the ground truth distribution while still preserving the necessary diversity of multi-modal predictions.

Experiments and Results

The performance of DifTraj was rigorously evaluated on two industry-standard benchmarks: the ETH-UCY dataset and the Stanford Drone Dataset (SDD). These datasets encompass a wide variety of real-world scenarios, from sparse sidewalk movement to dense campus interactions.

Quantitative Performance

DifTraj established a new state-of-the-art (SOTA) across both benchmarks:

  • ETH-UCY: The model achieved a remarkable 19.0% improvement in Average Displacement Error (ADE) and a 24.2% improvement in Final Displacement Error (FDE) compared to the previous leading model, LED.
  • SDD: DifTraj outperformed existing methods with a 5.2% improvement in ADE and an 8.3% improvement in FDE.

Qualitative Analysis

Visual comparisons highlight the model's ability to handle complex social scenes. In environments with dense crowds, such as the "UNIV" subset of ETH-UCY, DifTraj produces trajectories that are significantly closer to the actual ground truth than competing models. While other generative models often struggle with social navigation or goal-oriented consistency, DifTraj's paths remain focused on the predicted destination while naturally navigating around others.

Qualitative comparison of DifTraj against the state-of-the-art LED model across five ETH-UCY datasets.

Conclusion

DifTraj demonstrates that integrating intrinsic intention with extrinsic interaction significantly enhances the accuracy and reliability of multi-modal trajectory prediction. By anchoring diffusion processes with predicted goals and enforcing sample consistency, the model overcomes the instability and drift common in previous generative approaches. Future research will focus on refining the interpretation of ambiguous intentions and extending this framework to related tasks such as multi-object visual tracking and video segmentation.

Summary updated

BibTeX
@inproceedings{liu2024diftraj,
  author = {Yanghong Liu and Xingping Dong and Yutian Lin and Mang Ye},
  title = {{DifTraj: Diffusion Inspired by Intrinsic Intention and Extrinsic Interaction for Multi-Modal Trajectory Prediction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure