Abstract
MAP-Former introduces a novel motion prediction architecture that explicitly models the statistical dependencies between pairs of agents. By outputting joint Gaussian probability distributions for agent-pairs, the model provides a mathematically grounded framework to quantify interactivity and improve risk assessment in complex traffic scenarios. Evaluated on the rounD dataset, MAP-Former demonstrates significant improvements in joint prediction accuracy and reliability compared to standard baselines.
Introduction: The Challenge of Social Motion Prediction
In autonomous driving, predicting the future is inherently difficult because traffic is a "social" process. The movement of one agent, such as a car swerving, directly influences the behavior of nearby agents, like a cyclist braking. Most current motion prediction models provide "marginal" predictions, which quantify the uncertainty of a single agent's path in isolation. While some "joint" models predict the entire scene, they rarely provide explicit information about how the movement of Agent A is statistically linked to Agent B.
Without understanding these dependencies (covariance), risk assessment modules struggle to accurately calculate the probability of collisions or near-misses. MAP-Former (Multi-Agent-Pair Transformer) addresses this gap by bridging the divide between raw trajectory data and the nuanced relational data required for safe motion planning.
Methodology: The MAP-Former Architecture
The MAP-Former architecture is designed to capture both individual agent history and complex scene interactions through four primary components:
- Temporal Encoder (TEnc): A Transformer-based module that embeds the history of all agents into a high-dimensional space.
- Spatial and Interaction Encoder (SaIEnc): This module captures the scene context, including road geometry and agent relationships. The authors evaluated both Graph Neural Network (GNN) and Factorized Transformer implementations.
- Factorized Transformer Decoder: This component aggregates temporal and spatial data, utilizing cross-attention to update learned embeddings for future time steps.
- Multihead Prediction: The core innovation of the model. It features two sets of heads: one for standard multi-modal trajectories and another that predicts the parameters of a 4x4 covariance matrix for every agent-pair.
To ensure these predicted covariance matrices are mathematically valid (symmetric and positive-definite), the authors utilize a Cholesky decomposition ($LDL^T$) formulation. The model is trained using a Multivariate Gaussian Negative Log Likelihood (MGNNL) loss, forcing the production of probability distributions that accurately fit ground truth data.
Datasets and Evaluation
The study primarily utilizes the rounD dataset, which contains drone-captured trajectories from German roundabouts. Roundabouts serve as "interaction hotspots" where agents must constantly negotiate space, making them ideal for testing joint dependency predictions. The authors focused on this dataset to prioritize social awareness and agent-pair relations over simple displacement error minimization typically emphasized in leaderboard-centric datasets like Waymo Open Motion.
Key Insights: Quantifying Interactivity
A primary benefit of MAP-Former is the ability to visualize "interactivity" as a heatmap. By summing the absolute values of the cross-covariance blocks in the predicted matrices, the model reveals statistical dependencies. Agents in close proximity or in "leader-follower" roles exhibit significantly higher dependency than those further apart.
Furthermore, the model is scene-centric, predicting the evolution of the entire scene simultaneously. This allows any agent to utilize the data to understand the risks posed by others. The mathematical constraints imposed by the Cholesky decomposition ensure that the output is always a valid Gaussian probability density function (PDF).
Results
The "Full" MAP-Former, utilizing the Transformer-based spatial encoder, outperformed all baselines in joint prediction metrics:
- Accuracy: At a 3-second horizon, MAP-Former achieved a minSADE of 0.52m, significantly outperforming the CNN joint baseline (1.46m).
- Reliability: The Scene-Centric Miss Rate (SMR)—which records a failure if even one agent in the scene is predicted incorrectly—was 0.75 for MAP-Former at 3s, compared to 1.00 for the CNN baseline.
- Competitive Performance: Despite the increased difficulty of joint prediction, MAP-Former remained competitive with state-of-the-art marginal models like N-ODE2.
Limitations and Future Directions
While MAP-Former provides a robust framework for joint prediction, it faces challenges in computational complexity. Predicting covariance matrices for every pair in a crowded scene scales at $O(N^2)$, which may become expensive as the number of agents increases. Additionally, while the paper demonstrates the ability to predict these joint PDFs, future work is needed to integrate these predictions into active motion planners for real-time decision-making. The authors plan to conduct a more comprehensive statistical analysis of how these agent-pair correlations evolve over longer time horizons in subsequent research.
Summary updated