Deep Kinematic Models for Kinematically Feasible Vehicle Trajectory Predictions

Henggang Cui, Thi Nguyen, Fang-Chieh Chou, Tsung-Han Lin, Jeff Schneider, David Bradley, Nemanja DjuricICRA 2020 · DOI

TL;DR

This paper introduces a novel deep learning approach that integrates vehicle kinematics directly into trajectory prediction models for self-driving cars. By ensuring predictions are physically plausible, their method significantly improves accuracy and safety compared to existing state-of-the-art models that predict trajectories without considering these constraints.

What Problem Does This Paper Address?

Self-driving vehicles (SDVs) hold immense promise for improving traffic safety and efficiency. A crucial component for their safe operation is the ability to accurately predict the future movements of surrounding vehicles. Traditional deep learning methods for trajectory prediction often treat vehicles as simple points and learn motion constraints implicitly from data. However, this can lead to predictions that are kinematically impossible (e.g., turning too sharply for a vehicle's physical limitations), which can have serious consequences for an SDV's planning and decision-making. This paper tackles the problem of generating kinematically feasible trajectory predictions for surrounding vehicles.

Methods

The core contribution is the Deep Kinematic Model (DKM). It builds upon existing deep learning trajectory prediction frameworks (specifically, a CNN-based approach using bird's-eye view rasters) by introducing a kinematic layer. This layer is placed between the deep backbone network and the output layer. Instead of directly outputting future positions, the DKM's kinematic layer predicts control inputs (longitudinal acceleration and steering angle) at each future time step. These predicted controls are then used to roll out the vehicle's state (position, velocity, heading) using a standard two-axle vehicle kinematic model. This ensures that all predicted trajectories inherently satisfy vehicle dynamics constraints. The entire DKM is trained end-to-end using a loss function that penalizes deviations from ground truth trajectories, similar to prior work, but now with the added benefit of guaranteed kinematic feasibility.

Datasets

The authors used a large-scale, real-world dataset collected from a fleet of SDVs, comprising 240 hours of driving data at a 10Hz rate. This data was processed to generate tracked vehicle detections and their states (position, velocity, acceleration, heading, turning rate), along with map information. This dataset was split into training, validation, and testing sets with a 3:1:1 ratio.

Key Insights and Ideas

  • Kinematics are Crucial for Vehicle Prediction: Unlike pedestrians, vehicles have strict physical limitations (e.g., minimum turning radius) that cannot be ignored. Simply learning from data without explicit kinematic constraints leads to unrealistic and potentially dangerous predictions.
  • Combining Deep Learning and Physics: The paper demonstrates the power of integrating domain-specific physical models (vehicle kinematics) into deep learning architectures. This "best of both worlds" approach leverages the learning capacity of deep networks while ensuring outputs adhere to fundamental physical laws.
  • Control Prediction is Key: Predicting control inputs (acceleration and steering) rather than just positions allows for a more principled way to generate kinematically feasible trajectories.
  • Generalizability: The proposed kinematic layer is designed to be agnostic to the underlying deep learning architecture, suggesting it can be integrated into various existing motion prediction models.

Key Results

  • Improved Accuracy: DKM significantly reduces heading errors compared to unconstrained models, with improvements of over 2x in Wasserstein distance for turning rate distributions.
  • Guaranteed Feasibility: DKM guarantees kinematically feasible trajectories, eliminating the "infeasible" predictions seen in other methods. The paper reports 0% infeasible trajectories for DKM, compared to up to 30% for some unconstrained models.
  • State-of-the-Art Performance: DKM outperforms existing state-of-the-art methods in terms of both position and heading prediction accuracy, particularly at longer prediction horizons.
  • Qualitative Improvements: Case studies show DKM producing smoother and more realistic acceleration and heading profiles, especially during maneuvers like braking.

Limitations and Future Work

  • Model Generalizability: While the kinematic layer is general, the paper primarily evaluates it with a specific CNN-based backbone. Further exploration with other architectures (e.g., RNNs, GANs) and learning paradigms (e.g., imitation learning) is suggested.
  • Kinematic Parameter Learning: The paper uses fixed kinematic parameters (e.g., vehicle length) derived from tracked states. The authors note that these parameters could potentially be learned by the network.
  • Vehicle Types: The current implementation focuses on two-axle vehicles. Extending the model to handle other vehicle types (e.g., trucks, motorcycles) would require adapting the kinematic model.
  • Real-time Testing: While the paper mentions successful onboard testing, further details on real-time performance and computational efficiency would be valuable.

Summary updated

BibTeX
@inproceedings{cui2020deep,
  author = {Henggang Cui and Thi Nguyen and Fang-Chieh Chou and Tsung-Han Lin and Jeff Schneider and David Bradley and Nemanja Djuric},
  title = {{Deep Kinematic Models for Kinematically Feasible Vehicle Trajectory Predictions}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
  doi = {10.1109/icra40945.2020.9197560},
}
Figure