Introducing Probabilistic Bézier Curves for N-Step Sequence Prediction

Ronny Hug, Wolfgang Hübner, Michael ArensAAAI 2020 · DOI

Predicting where a person will walk or how a skeleton will move isn't just about finding the most likely path—it’s about understanding that there are often multiple, equally valid futures. In a paper presented at AAAI-20, researchers from the Fraunhofer Institute (IOSB) introduced a novel way to handle this uncertainty: the $\mathcal{N}$-Curve.

The Challenge of Multi-modal Prediction

Sequence prediction—predicting $n$ future steps based on $m$ past observations—is a cornerstone of autonomous driving and robotics. However, it faces two major hurdles:

  1. Multi-modality: In the real world, there isn't just one "correct" future. A car at an intersection might turn left, right, or go straight. Standard models often "blur" these options together, predicting a physically impossible path straight into a wall.
  2. Computational Efficiency: Many current models are autoregressive, meaning they predict one step at a time. To get a sense of uncertainty, you have to run thousands of Monte Carlo simulations, which is too slow for real-time planning.

The authors sought a model that is inherently smooth, captures multiple possible "modes" of movement, and provides a full probabilistic prediction for the entire future sequence all at once.

Methods: The $\mathcal{N}$-Curve

The core innovation is the $\mathcal{N}$-Curve (Probabilistic Bézier Curve). Traditionally used in computer graphics, Bézier curves are smooth paths defined by a set of control points.

The relationship between Gaussian control points (left) and the resulting continuous-time probabilistic Bézier curve (right).

In this probabilistic twist, the authors define the control points as Gaussian random variables. Because a linear combination of Gaussians remains a Gaussian, every single point along the resulting curve is also a Gaussian distribution. This transforms the Bézier curve into a continuous-time stochastic process. To handle multi-modality, they use a Mixture Density Network (MDN). An LSTM encoder processes past observations and feeds into a neural network that outputs the parameters (weights, means, and variances) for multiple $\mathcal{N}$-Curves simultaneously.

Key Insights and Results

By predicting the parameters of a curve rather than individual points, the model is forced to produce smooth outputs, effectively constraining the search space to physically plausible, continuous paths.

A toy example demonstrating how the model represents uncertainty and multi-modal paths in a 1D sequence.

A fascinating insight from their toy examples showed the "Superfluous Component" problem: if the model is given more "modes" (curves) than the data actually has, it doesn't always turn the extras off. Instead, it might overlap multiple curves to represent a single path, suggesting that weight regularization is a necessary component for these types of mixture models.

Performance Highlights

The $\mathcal{N}$-Curve MDN demonstrated state-of-the-art performance across two distinct tasks:

  • Trajectory Prediction: On the Stanford Drone Dataset (SDD), the model achieved a lower Final Displacement Error (FDE) and better Negative Log-Likelihood (NLL) than established baselines like Particle LSTMs. In the 'hyang' scene, it achieved an FDE of 0.088 compared to 0.129 for the Particle LSTM.
  • Motion Capture: In the CMU Motion Capture Database, it outperformed both standard Multi-Layer Perceptrons (MLP) and Recurrent Gaussian Processes (RGP) in terms of Root Mean Square Error (RMSE), proving it can scale to high-dimensional skeleton data (59 dimensions).

Limitations and Future Work

While promising, the paper notes a few areas for growth. Currently, the model is defined for a fixed index set (e.g., predicting exactly 50 steps). The authors aim to extend this to a recurrent system that can handle variable-length sequences. Additionally, they suggest investigating non-linear variance interpolation to better model how uncertainty grows or shrinks over time, rather than the current linear weighting approach.

Summary updated

BibTeX
@inproceedings{hug2020introducing,
  author = {Ronny Hug and Wolfgang Hübner and Michael Arens},
  title = {{Introducing Probabilistic Bézier Curves for N-Step Sequence Prediction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i06.6576},
}
Figure