Abstract
Predicting human behavior is a difficult and crucial task required for motion planning. It is challenging in large part due to the highly uncertain and multi-modal set of possible outcomes in real-world domains such as autonomous driving. Beyond single MAP trajectory prediction, obtaining an accurate probability distribution of the future is an area of active interest. We present MultiPath, which leverages a fixed set of future state-sequence anchors that correspond to modes of the trajectory distribution. At inference, our model predicts a discrete distribution over the anchors and, for each anchor, regresses offsets from anchor waypoints along with uncertainties, yielding a Gaussian mixture at each time step. Our model is efficient, requiring only one forward inference pass to obtain multi-modal future distributions, and the output is parametric, allowing compact communication and analytical probabilistic queries. We show on several datasets that our model achieves more accurate predictions, and compared to sampling baselines, does so with an order of magnitude fewer trajectories.
Introduction
Predicting the future behavior of agents such as cars, pedestrians, and cyclists is critical for safe autonomous driving. However, human behavior is inherently stochastic and multi-modal. For example, a car approaching an intersection might turn left, turn right, or go straight.
Previous approaches generally fall into two categories. Deterministic models predict a single most-likely future trajectory, which often results in predicting an "average" path that is physically impossible in multi-modal scenarios. Stochastic sampling models, such as GANs or CVAEs, generate diverse sets of future samples but often suffer from mode collapse and introduce non-determinism, making it computationally difficult to answer exact probabilistic queries.
The MultiPath Method
MultiPath frames trajectory prediction as a hierarchical problem, outputting a parametric Gaussian Mixture Model (GMM) in a single forward inference pass. The architecture uses a top-down, bird's-eye view representation of the scene to extract features.
The core innovation is the use of trajectory anchors. Instead of predicting paths from scratch, the model relies on a fixed set of $K$ anchor trajectories discovered via k-means clustering on training data. The model factorizes uncertainty into two parts:
- Intent Uncertainty: A discrete softmax distribution over the $K$ anchors.
- Control Uncertainty: For each anchor, the model regresses Gaussian parameters (mean offsets and covariance matrices) at each future time step to refine the path.
Experimental Results
The model was evaluated on several datasets, including a massive real-world autonomous driving dataset, the Stanford Drone Dataset (SDD), and the CARLA simulator.
- Real-World Driving: MultiPath achieved a log-likelihood of 4.37 nats, significantly beating a single-trajectory regression baseline. For the top 5 predictions (
minADE_5), MultiPath achieved an error of 0.58 meters, outperforming the CVAE sampling baseline of 1.82 meters. - Stanford Drone Dataset: MultiPath achieved a
minADE_5of 17.51 pixels, outperforming previous state-of-the-art methods like DESIRE (30.78 pixels). - CARLA: MultiPath matched or slightly beat state-of-the-art models despite its simpler single-pass architecture.
Key Insights and Limitations
A critical insight is that heading normalization—rotating agent-centric features—significantly improves performance. Additionally, parametric distributions allow for exact probabilistic queries, which is preferable for safety-critical systems compared to random sampling.
However, the model assumes time steps are conditionally independent given an anchor, which is a simplification. Performance also depends heavily on the quality and number of pre-defined anchors ($K$). Future work could explore recurrent structures to model conditional next-step dependencies and alternative anchor assignment strategies during training.
Summary updated