Rethinking Trajectory Prediction via "Team Game"

Zikai Wei, Xinge Zhu, Bo Dai, Dahua LinarXiv 2022 · DOI · arXiv

Abstract

Trajectory prediction is a critical component for autonomous systems, yet traditional models often overlook the hierarchical nature of group dynamics. This paper introduces Interactive Group Consensus (IGC), a framework that treats multi-agent movement as a "team game." By explicitly modeling the interplay between group-level strategies and individual preferences, we propose the Spatial-Temporal Graph Variational RNN with an Interactive Hierarchical Latent Space (IHVRNN). Our approach achieves state-of-the-art performance in both competitive sports and collaborative pedestrian environments by capturing the explicit conjecture and negotiation inherent in group movement.

Introduction

Predicting agent movement—whether for pedestrians, vehicles, or athletes—is essential for safe and efficient autonomous navigation. Most existing models treat agents as independent entities that interact locally to avoid collisions. However, real-world movement is frequently governed by a hierarchy of intent. For instance, soccer players move according to a team strategy that reacts to the opponent's formation, and pedestrians often move in implicit groups.

Conceptual overview of the Interactive Group Consensus (IGC) process.

Current methods typically model these interactions implicitly, failing to capture the strategic "conjecture" and "negotiation" between groups and their members.

Methodology: The IHVRNN Framework

The core of our approach is the Interactive Hierarchical Latent Space (IHVRNN), which decomposes movement into two levels:

  1. Level 1: Group Consensus (Strategy): This level captures the collective goal of a group. It is inherently interactive; the model conjectures about the opponent's strategy to refine its own.
  2. Level 2: Individual Behavior: This level captures personal preferences and fine-grained movements, which are unified by the overarching group consensus.
The IHVRNN framework models trajectory prediction as a team game through (a) conjecture on opponent strategy, (b) reasoning on own group consensus, and (c) the overall interactive process.

The IHVRNN pipeline comprises four modules:

  • Updating Module: Summarizes historical states for individuals and groups.
  • Team Game Priors: Uses game-theoretic intuition to model strategy formation through cross-group conjecture.
  • Team Game Encoders: Maps trajectories into the hierarchical latent space.
  • Trajectory Decoder: Generates future paths by integrating group strategy, individual preference, and social impacts via Graph Attention Networks.

Furthermore, we implement Scope-Changing Social Masks, allowing the model to adjust its attention span based on the context—from global field awareness in sports to local proximity in pedestrian crowds.

Experimental Evaluation

We evaluated IHVRNN on two distinct datasets:

  • Team Sports: The Europa League dataset, featuring tracking data from 45 professional soccer matches.
  • Pedestrians: The ETH and UCY benchmarks, covering various crowd scenarios.

Key Results

IHVRNN consistently outperformed baselines such as Social-LSTM, Social-GAN, and Trajectron++.

  • Soccer Performance: On the Europa League dataset, IHVRNN achieved a Mean Squared Error (MSE) of 0.0038, outperforming the best baseline, Role-Assign (0.0041).
  • Pedestrian Performance: On ETH/UCY, IHVRNN reached the lowest Average Displacement Error (0.47) and Final Displacement Error (1.00).
  • Ablation Study: Removing the interactive component or the hierarchical structure led to significant performance drops, validating the importance of multi-level strategic modeling.

Discussion and Conclusion

This work shifts the paradigm of trajectory prediction from physics-based collision avoidance to sociology-inspired game theory. By treating agents as members of a collective with shared interests, the model prevents the generation of strategically nonsensical trajectories. Future work will focus on modeling dynamic group memberships and scaling the approach to even larger datasets.

Summary updated

Abstract

To accurately predict trajectories in multi-agent settings, e.g. team games, it is important to effectively model the interactions among agents. Whereas a number of methods have been developed for this purpose, existing methods implicitly model these interactions as part of the deep net architecture. However, in the real world, interactions often exist at multiple levels, e.g. individuals may form groups, where interactions among groups and those among the individuals in the same group often follow significantly different patterns. In this paper, we present a novel formulation for multi-agent trajectory prediction, which explicitly introduces the concept of interactive group consensus via an interactive hierarchical latent space. This formulation allows group-level and individual-level interactions to be captured jointly, thus substantially improving the capability of modeling complex dynamics. On two multi-agent settings, i.e. team sports and pedestrians, the proposed framework consistently achieves superior performance compared to existing methods.
BibTeX
@misc{wei2022rethinking,
  author = {Zikai Wei and Xinge Zhu and Bo Dai and Dahua Lin},
  title = {{Rethinking Trajectory Prediction via "Team Game"}},
  year = {2022},
  eprint = {2210.08793},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.08793},
}
Figure