When an autonomous vehicle approaches a four-way stop, it faces a fundamental challenge: the human driver across the intersection might accelerate to go first, or they might tap their brakes to yield. These are not just slightly different versions of the same path; they are distinct, mutually exclusive "modes" of behavior.
Traditional regression models often struggle here, producing a single "average" trajectory that splits the difference—resulting in a path that drives straight into the middle of the intersection at an unrealistic, half-hearted speed. To solve this, researchers have converged on a powerful architecture: the Conditional Variational Autoencoder (CVAE) with discrete latent variables.
The Multimodality Problem
In human-robot interaction (HRI), "multimodality" refers to the reality that there are multiple, equally plausible futures. As Edward Schmerling, Karen Leung, and Marco Pavone argue in Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction, capturing these modes is essential for safe negotiation. If a robot only predicts the most likely outcome, it remains blind to "long-tail" risks—like a pedestrian suddenly darting across the street.
Discretizing Intent
The breakthrough in recent work is the use of discrete latent variables to represent high-level intent. Instead of a continuous "style" variable, the model is forced to choose from a set of categorical modes.
- Unsupervised Clustering: During training, the discrete latent space acts as an unsupervised clustering mechanism. As shown in Trajectron++ by Boris Ivanovic and colleagues, the model automatically learns to group trajectories into semantically meaningful categories like "turning left," "braking," or "accelerating."
- Preventing Mode Collapse: Continuous latent spaces often suffer from "posterior collapse," where the model ignores the latent variable and produces a single, blurry average. Discrete variables, often implemented via the Gumbel-Softmax trick as seen in GroupNet by Siheng Chen, force the generator to produce distinct, non-overlapping paths.
- Interpretability: Because each latent state corresponds to a specific behavioral mode, the model becomes "interpretable." A planner can look at the latent variable and understand why a trajectory was generated (e.g., "The model is currently simulating the 'yielding' mode").
From Prediction to Planning
Capturing these modes isn't just about better "guessing"; it’s about better "doing." In MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control, Adrien Gaidon and his team demonstrate that by representing these modes as distinct dynamical systems, a robot can optimize its own path while accounting for how a human might respond to its actions.
This approach even extends to "seeing" through walls. Masha Itkina and Katherine Rose Driggs-Campbell show in Multi-Agent Variational Occlusion Inference Using People as Sensors that by observing how a human driver behaves, a discrete latent model can infer multiple hypotheses about what might be hidden in a blind spot, allowing the robot to reason about a distribution of risks rather than a single, potentially fatal assumption.