Predicting where a person will walk or how a skeleton will move isn't just about finding the most likely path—it’s about understanding that there are often multiple, equally valid futures. In a paper presented at AAAI-20, researchers from the Fraunhofer Institute (IOSB) introduced a novel way to handle this uncertainty: the $\mathcal{N}$-Curve.
The Challenge of Multi-modal Prediction
Sequence prediction—predicting $n$ future steps based on $m$ past observations—is a cornerstone of autonomous driving and robotics. However, it faces two major hurdles:
- Multi-modality: In the real world, there isn't just one "correct" future. A car at an intersection might turn left, right, or go straight. Standard models often "blur" these options together, predicting a physically impossible path straight into a wall.
- Computational Efficiency: Many current models are autoregressive, meaning they predict one step at a time. To get a sense of uncertainty, you have to run thousands of Monte Carlo simulations, which is too slow for real-time planning.
The authors sought a model that is inherently smooth, captures multiple possible "modes" of movement, and provides a full probabilistic prediction for the entire future sequence all at once.
Methods: The $\mathcal{N}$-Curve
The core innovation is the $\mathcal{N}$-Curve (Probabilistic Bézier Curve). Traditionally used in computer graphics, Bézier curves are smooth paths defined by a set of control points.
In this probabilistic twist, the authors define the control points as Gaussian random variables. Because a linear combination of Gaussians remains a Gaussian, every single point along the resulting curve is also a Gaussian distribution. This transforms the Bézier curve into a continuous-time stochastic process. To handle multi-modality, they use a Mixture Density Network (MDN). An LSTM encoder processes past observations and feeds into a neural network that outputs the parameters (weights, means, and variances) for multiple $\mathcal{N}$-Curves simultaneously.
Key Insights and Results
By predicting the parameters of a curve rather than individual points, the model is forced to produce smooth outputs, effectively constraining the search space to physically plausible, continuous paths.
A fascinating insight from their toy examples showed the "Superfluous Component" problem: if the model is given more "modes" (curves) than the data actually has, it doesn't always turn the extras off. Instead, it might overlap multiple curves to represent a single path, suggesting that weight regularization is a necessary component for these types of mixture models.
Performance Highlights
The $\mathcal{N}$-Curve MDN demonstrated state-of-the-art performance across two distinct tasks:
- Trajectory Prediction: On the Stanford Drone Dataset (SDD), the model achieved a lower Final Displacement Error (FDE) and better Negative Log-Likelihood (NLL) than established baselines like Particle LSTMs. In the 'hyang' scene, it achieved an FDE of 0.088 compared to 0.129 for the Particle LSTM.
- Motion Capture: In the CMU Motion Capture Database, it outperformed both standard Multi-Layer Perceptrons (MLP) and Recurrent Gaussian Processes (RGP) in terms of Root Mean Square Error (RMSE), proving it can scale to high-dimensional skeleton data (59 dimensions).
Limitations and Future Work
While promising, the paper notes a few areas for growth. Currently, the model is defined for a fixed index set (e.g., predicting exactly 50 steps). The authors aim to extend this to a recurrent system that can handle variable-length sequences. Additionally, they suggest investigating non-linear variance interpolation to better model how uncertainty grows or shrinks over time, rather than the current linear weighting approach.
Summary updated