Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling

Chen Tang, Wei Zhan, Masayoshi Tomizuka

Introduction

Predicting the future trajectories of multiple interacting agents is a fundamental challenge for autonomous systems. While Variational Autoencoders (VAEs) are widely used to capture the multi-modal nature of these interactions, they often suffer from a phenomenon known as posterior collapse. This research identifies a specific variant called Social Posterior Collapse, where the model learns to ignore the movements of surrounding agents, relying solely on an agent's own history and ground-truth future during training. This leads to a failure in generalization when the model is deployed in novel, complex environments where social context is critical for safety.

Methodology

To address social posterior collapse, the authors propose a framework designed to enforce social awareness through three primary mechanisms:

  1. Conditional VAE (CVAE) Formulation: The standard VAE is replaced with a CVAE that explicitly conditions the latent space on historical social context, ensuring that latent variables are grounded in the environment's state.
  2. Auxiliary Prediction Task: An additional decoder is introduced that is denied access to the ground-truth future during training. This "stick" approach forces the model to extract and utilize information from the social context and latent variables to minimize prediction error.
  3. Sparse Graph Attention Message-Passing (Sparse-GAMP): By employing $\alpha$-entmax instead of standard Softmax, the model can assign exactly zero weight to irrelevant neighbors. This sparsity allows for the definition of the Agent Ratio (AR) metric, which quantifies the percentage of surrounding agents the model actively considers.
Proposed model architecture featuring Sparse-GAMP and auxiliary prediction tasks to mitigate social posterior collapse.

Experimental Results

The proposed "Social-CVAE" was evaluated against standard VAE and CVAE baselines on the INTERACTION, Argoverse, and ETH/UCY datasets.

Quantifying Social Awareness

The Agent Ratio (AR) metric served as a diagnostic tool for social collapse. In the INTERACTION dataset's Highway Merging (HM) scene, the vanilla VAE exhibited an AR of approximately 8.68%, indicating significant social blindness. In contrast, the Social-CVAE maintained an AR of 15.5%, demonstrating a higher degree of social integration.

Accuracy and Generalization

The benefits of mitigating social posterior collapse are most evident in generalization tasks. On the INTERACTION Generalization (G) track, the Social-CVAE reduced the Final Displacement Error (FDE) from 3.85 (standard VAE) to 2.70. Similar improvements were observed in the Argoverse dataset, where the model achieved a minADE of 1.59 (vs. 1.86 for VAE) and a minFDE of 3.52 (vs. 4.17).

Performance comparison on the INTERACTION dataset across different scenes and tracks, highlighting improvements in Agent Ratio (AR) and displacement errors.

Discussion and Conclusion

The findings suggest that models are inherently "lazy" solvers, opting for the path of least resistance by ignoring complex social interactions if training losses can be minimized otherwise. While the proposed fixes significantly improved performance for vehicle interactions, they were less effective for pedestrian movement in the ETH/UCY dataset, likely due to the less constrained nature of pedestrian paths over short horizons. Future work may explore autoregressive decoders to further enhance multi-step prediction robustness.

Summary updated

Abstract

Multi-agent behavior modeling and trajectory forecasting are crucial for the safe navigation of autonomous agents in interactive scenarios. Variational Autoencoder (VAE) has been widely applied in multi-agent interaction modeling to generate diverse behavior and learn a low-dimensional representation for interacting systems. However, existing literature did not formally discuss if a VAE-based model can properly encode interaction into its latent space. In this work, we argue that one of the typical formulations of VAEs in multi-agent modeling suffers from an issue we refer to as social posterior collapse, i.e., the model is prone to ignoring historical social context when predicting the future trajectory of an agent. It could cause significant prediction errors and poor generalization performance. We analyze the reason behind this under-explored phenomenon and propose several measures to tackle it. Afterward, we implement the proposed framework and experiment on real-world datasets for multi-agent trajectory prediction. In particular, we propose a novel sparse graph attention message-passing (sparse-GAMP) layer, which helps us detect social posterior collapse in our experiments. In the experiments, we verify that social posterior collapse indeed occurs. Also, the proposed measures are effective in alleviating the issue. As a result, the model attains better generalization performance when historical social context is informative for prediction.
BibTeX
@inproceedings{tang2021exploring,
  author = {Chen Tang and Wei Zhan and Masayoshi Tomizuka},
  title = {{Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=iNUKmzaL-M5},
}
Figure