Temporally-Continuous Probabilistic Prediction using Polynomial Trajectory Parameterization

Zhaoen Su

TL;DR

This paper introduces a novel way to represent actor trajectories in self-driving scenarios: instead of predicting discrete future points, they propose parameterizing the entire trajectory with polynomials. This approach leads to smoother, more physically realistic predictions and improved accuracy, especially at times between predicted waypoints.

What Problem Does This Paper Address?

Predicting the future movements of other actors (vehicles, pedestrians, cyclists) is crucial for safe autonomous driving. The common method is to predict a sequence of future positions (waypoints) at specific time intervals. However, this approach has limitations:

  • Unrealistic Motion: Linear interpolation between waypoints can lead to jerky, physically implausible motions, especially concerning acceleration and jerk.
  • Approximation Errors: Predictions are only made at discrete time steps, and interpolating between them can introduce errors, particularly for dynamic maneuvers.
  • Computational Inefficiency: To get finer temporal resolution, one needs to predict many more waypoints, increasing computational load.

The authors aim to address these issues by proposing a representation that is continuous in time, leading to more accurate and physically realistic predictions without requiring explicit physics constraints.

Methods

The core innovation is representing actor trajectories using polynomial parameterization. Instead of directly regressing future waypoints, the model regresses the coefficients of polynomials that describe the actor's position and orientation over time.

  • Polynomial Trajectory Parameterization: Each component of the actor's state (e.g., x-position, y-position, yaw angle) is modeled as a polynomial function of time. This allows for a continuous representation of the trajectory.
  • Probabilistic Prediction: The model also predicts parameters for a probability distribution (specifically, a Laplace distribution) for each component, capturing uncertainty. The mean of this distribution is parameterized by a polynomial, and the diversity parameter (related to the spread of the distribution) is also parameterized by a polynomial.
  • Leveraging Existing Models: The proposed representation is designed to be a drop-in replacement for existing waypoint-based prediction models. The authors adapt the MultiXNet architecture to demonstrate this.

Datasets

  • nuScenes: An open-sourced dataset used for evaluation, with 7000 scenes and 2Hz annotations.
  • Uber ATG In-house Data: A larger, proprietary dataset with 14000 scenes and 10Hz annotations, used for more robust evaluation due to finer temporal resolution and lower metric variance.

Key Insights and Ideas

  • Smoothness is Key: Polynomials inherently enforce smoothness, leading to more realistic higher-order derivatives (like acceleration and jerk) compared to linear interpolation between waypoints.
  • Bridging the Temporal Gap: The polynomial representation excels at predicting accurate trajectories at interpolated time-points (i.e., times between the supervised training intervals), where linear interpolation struggles.
  • Implicit Physical Realism: The polynomial representation naturally enforces physical realism without needing explicit physics constraints or regularization terms in the loss function. This is a significant advantage over methods that require complex physics-aware model design.
  • Low-Order Polynomials Suffice: Surprisingly, low-order polynomials (degree 2 or 3) are often sufficient to capture actor motion accurately, offering a compact representation.
  • Benefits for Rare Actors: The regularization effect of polynomial representations appears particularly beneficial for predicting the motion of less common actors like bicyclists.

Key Results

  • Improved Accuracy: The polynomial representation achieves comparable or better accuracy than waypoint-based methods, especially at interpolated time-points. For instance, in Table IV, P2 (polynomial of degree 2) shows better performance at 1s and 3s (interpolated times) compared to linearly interpolated WP predictions.
  • Physical Realism: Figure 3 demonstrates that polynomial-based predictions exhibit distributions of acceleration and lateral acceleration much closer to ground truth labels than waypoint-based predictions, which often show infeasible values.
  • Efficiency: For an 8-second prediction, a polynomial model might regress around 20 coefficients, whereas a waypoint model could regress 648 values, indicating a more compact representation.
  • Comparable Performance: In many scenarios, especially with sufficient supervision, the polynomial approach performs similarly to waypoint methods on the supervised time-points, validating its effectiveness as a replacement.

Limitations and Future Work

  • Hard Maneuvers: The paper notes that high approximation errors can occur on maneuvers with high jerk or abrupt changes in motion. While low-order polynomials might not hurt in these cases, higher-order polynomials or splines could be explored.
  • Model Adaptation: While the paper shows the representation can be integrated into existing models like MultiXNet, further exploration of how different model architectures interact with this representation could be beneficial.
  • Supervision Interval: The effectiveness of polynomial representations is highlighted when supervision intervals are large, suggesting that further studies with even larger intervals or unsupervised learning could be interesting.
  • Pedestrian Motion: The paper notes that polynomial models perform similarly to waypoint models for pedestrians, suggesting that second-order states might not be well-captured or are less critical for pedestrian prediction. Further investigation into this could be valuable.

Summary updated

Abstract

A commonly-used representation for motion prediction of actors is a sequence of waypoints (comprising positions and orientations) for each actor at discrete future time-points. While regressing waypoints is simple and flexible, it can exhibit unrealistic higher-order derivatives (such as acceleration) and approximation errors at intermediate time steps. To address this issue we propose a general representation for temporally-continuous probabilistic trajectory prediction that regresses polynomial parameterization coefficients. We evaluate the proposed representation on supervised trajectory prediction tasks using two large self-driving data sets. The results show realistic higher-order derivatives and better accuracy at interpolated time-points, as well as the benefits of the inferred noise distributions over the trajectories. Extensive experimental studies based on existing state-of-the-art models demonstrate the effectiveness of the proposed approach relative to other representations in predicting the future motions of vehicle, bicyclist, and pedestrian traffic actors.
BibTeX
@inproceedings{su2021temporally,
  author = {Zhaoen Su},
  title = {{Temporally-Continuous Probabilistic Prediction using Polynomial Trajectory Parameterization}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=ZhhX-8t9voH},
}
Figure