Mode Collapse Happens: Evaluating Critical Interactions in Joint Trajectory Prediction Models

Maarten Hugenholtz, Anna Mészáros, Jens Kober, Zlatan AjanovicarXiv 2025 · DOI · arXiv

Abstract

Predicting the future movements of multiple agents is a core challenge for autonomous vehicles. This paper introduces a novel evaluation framework to detect and quantify mode collapse in joint trajectory prediction models. By focusing on safety-critical interactions and using topological "homotopy" to classify them, the authors demonstrate that current state-of-the-art models frequently fail to predict feasible interaction alternatives. This "interaction-blindness" in standard metrics poses significant risks for downstream motion planning.

Introduction

In autonomous driving, predicting the future movement of surrounding agents is rarely a single-answer problem; it is fundamentally multimodal. For instance, a car at an intersection might turn left or go straight, and it might yield or accelerate to pass first.

Current models often suffer from mode collapse, where a model only predicts the most likely outcome (the "average" behavior) while ignoring less likely but still feasible alternatives. The authors argue that standard evaluation metrics like Average Displacement Error (ADE) are insufficient because they are "interaction-blind." A model can achieve a low error score by predicting a trajectory that is physically close to the ground truth, even if it fundamentally misses the logic of the interaction (e.g., predicting a car will pass first when it actually yields).

Comparison of marginal vs. joint prediction modes and their associated probabilities.

Methodology

The authors propose a framework that moves beyond simple distance-based errors by focusing on the topological structure of interactions:

  1. Safety-Critical Filtering: A "Path-Sharing" (PS) algorithm identifies pairs of agents whose paths will actually cross, filtering out non-critical scenarios like simple car-following.
  2. Homotopy Classification: To define an "interaction mode," the framework uses free-end homotopy. By calculating a "winding angle" between two agents, interactions are classified as either Clockwise (CW) or Counterclockwise (CCW), capturing the "who goes first" logic.
  3. Inevitable Homotopy State (IHS): This defines the point in time where physics and collision constraints leave only one possible interaction mode. Evaluation focuses on the window before this collapse.
Classification of common traffic interactions: merging, crossing, car-following, and passing.

New Metrics

  • Mode Collapse Rate: The percentage of time the model fails to predict a feasible (non-colliding) interaction mode.
  • Time-to-Correct/Covered ($\Delta T$): The lead time provided before the interaction is settled.
  • Temporal Consistency: Whether the model’s prediction of the interaction mode remains stable over time.

Experimental Setup

The study utilizes the nuScenes dataset. While the full dataset is vast, the authors' filtering method identified 351 safety-critical interaction pairs (primarily merging and crossing) for evaluation. This highlights the rarity of truly "interesting" interaction events in standard on-road datasets.

Results and Discussion

The evaluation revealed several critical insights into the performance of modern prediction models:

  • The "Constant Velocity" Paradox: Surprisingly, a simple Constant Velocity (CV) baseline often outperformed complex deep learning models in predicting the correct interaction mode. This suggests that an agent's current heading and speed already implicitly communicate their intent.
  • Distance Metrics are Deceptive: Models like AgentFormer (AF) can appear superior on traditional joint distance metrics (minADE/minFDE) while being significantly worse at capturing the diversity of interaction modes compared to simpler baselines.
  • Pervasive Mode Collapse: Even state-of-the-art models showed a mode collapse rate of approximately 70% in safety-critical interactions. They frequently failed to predict a feasible alternative even when that alternative was physically possible and non-colliding.
  • Late Awareness: Models often only "realize" the correct interaction mode very close to the event, leaving minimal time for a motion planner to react safely.

Conclusion

This research demonstrates that mode collapse is a real and pervasive issue in joint trajectory prediction. Current models often fail to provide the diversity of predictions necessary for safe autonomous navigation in interactive environments. Future work should focus on utilizing more diverse datasets, such as drone-captured data, and moving toward closed-loop simulations to fully understand the safety impacts of these predictive failures.

Summary updated

Abstract

Autonomous Vehicle decisions rely on multimodal prediction models that account for multiple route options and the inherent uncertainty in human behavior. However, models can suffer from mode collapse, where only the most likely mode is predicted, posing significant safety risks. While existing methods employ various strategies to generate diverse predictions, they often overlook the diversity in interaction modes among agents. Additionally, traditional metrics for evaluating prediction models are dataset-dependent and do not evaluate inter-agent interactions quantitatively. To our knowledge, none of the existing metrics explicitly evaluates mode collapse. In this paper, we propose a novel evaluation framework that assesses mode collapse in joint trajectory predictions, focusing on safety-critical interactions. We introduce metrics for mode collapse, mode correctness, and coverage, emphasizing the sequential dimension of predictions. By testing four multi-agent trajectory prediction models, we demonstrate that mode collapse indeed happens. When looking at the sequential dimension, although prediction accuracy improves closer to interaction events, there are still cases where the models are unable to predict the correct interaction mode, even just before the interaction mode becomes inevitable. We hope that our framework can help researchers gain new insights and advance the development of more consistent and accurate prediction models, thus enhancing the safety of autonomous driving systems.
BibTeX
@misc{hugenholtz2025mode,
  author = {Maarten Hugenholtz and Anna Mészáros and Jens Kober and Zlatan Ajanovic},
  title = {{Mode Collapse Happens: Evaluating Critical Interactions in Joint Trajectory Prediction Models}},
  year = {2025},
  eprint = {2506.23164},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.23164},
}
Figure