CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention

Julian Schmidt, Julian Jordan, Franz Gritschneder, Klaus Dietmayer

This paper presents CRAT-Pred (Crystal Attention Prediction), a novel approach to predicting how vehicles move in traffic without relying on high-definition (HD) maps. By combining techniques from material science with modern transformer architectures, the authors achieve state-of-the-art performance in "map-free" prediction while using significantly fewer parameters than competing models.

TL;DR

CRAT-Pred is a map-free trajectory prediction model that uses Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention to model social interactions between vehicles. It achieves state-of-the-art results on the Argoverse benchmark with a highly efficient architecture, proving that self-attention weights can serve as a measurable proxy for social interaction.

What Problem Does This Paper Address?

Most current state-of-the-art (SOTA) vehicle trajectory prediction models are "map-heavy"—they rely on HD maps to understand road geometry and traffic rules. However, HD maps are not always available, can be outdated, or may become unusable due to localization failures (e.g., poor GPS in a tunnel).

The challenge is: how do we predict a vehicle's future path accurately using only the observed movements of surrounding actors? This requires modeling "social interactions"—understanding how a trailing vehicle's behavior depends on the leading vehicle. While some "map-free" models exist, they often struggle to outperform simple baselines like constant velocity models or require massive amounts of parameters to capture complex dependencies.

Methods

CRAT-Pred uses a three-part architecture to process vehicle data:

  1. Input Encoder: A Long Short-Term Memory (LSTM) network encodes the past two seconds of a vehicle's motion (represented as discrete displacements) into a hidden state.
  2. Interaction Module: This is the core innovation. It uses two distinct mechanisms:
  3. Crystal Graph Convolutional Neural Networks (CGCNN): Borrowed from material science (where they model atomic bonds), these are used here to model the "bonds" between vehicles. Unlike standard graph convolutions, CGCNNs explicitly incorporate edge features—in this case, the relative distance between vehicles—to update vehicle representations.
  4. Multi-Head Self-Attention (MHSA): This layer allows the model to weigh the importance of different surrounding vehicles. It helps the model "pay attention" to the most relevant actors regardless of their distance.
  5. Output Decoder: A series of linear residual layers generate multiple possible future trajectories (multi-modality). The model is trained in two stages: first, a single-path backbone is trained, then additional decoders are added and trained using a "Winner-Takes-All" loss to encourage diverse, realistic predictions.

Datasets

The authors evaluated their model using the Argoverse Motion Forecasting Dataset, a industry-standard benchmark containing over 300,000 real-world driving sequences from Miami and Pittsburgh.

  • Task: Use 2 seconds of history to predict 3 seconds into the future.
  • Constraint: Although Argoverse provides HD maps, the authors intentionally ignored them to test the "map-free" capabilities of CRAT-Pred.

Key Insights and Ideas

  • Cross-Domain Inspiration: The use of CGCNNs suggests that the spatial relationships between vehicles in traffic share mathematical similarities with atomic structures in crystals. Specifically, the ability of CGCNNs to handle edge features (distances) is more effective for traffic than standard node-only graph networks.
  • Attention as an Interaction Score: The authors quantitatively prove that self-attention weights are not just a computational trick; they are a superior indicator of social interaction compared to simple Euclidean distance. Selecting the "top-N" most important vehicles based on attention weights leads to better prediction accuracy than simply selecting the "N-closest" vehicles.
  • Efficiency Matters: CRAT-Pred achieves its performance with roughly 515k parameters. For context, other high-performing models like LaneGCN use over 1 million, and WIMP uses over 20 million. This makes CRAT-Pred much more suitable for real-time deployment on embedded automotive hardware.

Key Results

  • Map-Free SOTA: On the Argoverse test set, CRAT-Pred achieved a minADE (minimum Average Displacement Error) of 1.82m, significantly outperforming the standard Argoverse map-free baselines (which ranged from 2.15m to 3.53m).
  • Competitive with Map-Based Models: Even without using any map information, CRAT-Pred's performance is surprisingly close to some models that do use maps, highlighting the power of social interaction modeling.
  • Interpretability: Qualitative results show that the attention mechanism correctly identifies critical actors, such as vehicles merging into a lane or those directly in the path of the target vehicle, even if they aren't the closest ones.

Limitations and Future Work

  • Lack of Geometric Constraints: Because the model is map-free, it may occasionally predict trajectories that are physically possible but "illegal" (e.g., driving onto a sidewalk or across a median) because it doesn't "see" those boundaries.
  • Future Integration: The authors suggest that while this model is designed to be map-free, the CGCNN/Attention architecture could potentially be extended to incorporate map features in the future to create an even more robust hybrid system.
  • Scenario Specificity: While tested on diverse urban data, the model's performance in extremely dense, chaotic environments (like unstructured intersections) remains an area for further exploration.

Summary updated

Abstract

Predicting the motion of surrounding vehicles is essential for autonomous vehicles, as it governs their own motion plan. Current state-of-the-art vehicle prediction models heavily rely on map information. In reality, however, this information is not always available. We therefore propose CRAT-Pred, a multi-modal and non-rasterization-based trajectory prediction model, specifically designed to effectively model social interactions between vehicles, without relying on map information. CRAT-Pred applies a graph convolution method originating from the field of material science to vehicle prediction, allowing to efficiently leverage edge features, and combines it with multi-head self-attention. Compared to other map-free approaches, the model achieves state-of-the-art performance with a significantly lower number of model parameters. In addition to that, we quantitatively show that the self-attention mechanism is able to learn social interactions between vehicles, with the weights representing a measurable interaction score. The source code is publicly available33Source code: https://github.com/schmidt-ju/crat-pred.
BibTeX
@inproceedings{schmidt2022crat,
  author = {Julian Schmidt and Julian Jordan and Franz Gritschneder and Klaus Dietmayer},
  title = {{CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=tizBYtoWgp},
}

Loading related content...

Figure