DDK: A Deep Koopman Approach for Longitudinal and Lateral Control of Autonomous Ground Vehicles

Yongqian Xiao, Xinglong Zhang, Xin Xu, Yang Lu, Junxiang Li

TL;DR

This paper introduces Deep Direct Koopman (DDK), a novel deep learning approach for modeling complex vehicle dynamics. DDK leverages Koopman operator theory to learn a linear representation of nonlinear vehicle behavior, enabling more effective trajectory tracking through a DDK-MPC controller.

What Problem Does This Paper Address?

Autonomous driving systems require accurate models of vehicle dynamics for tasks like trajectory prediction, motion planning, and control. However, real-world vehicle dynamics are highly nonlinear, making them challenging to model with traditional methods. Existing deep learning approaches often produce "black box" models that lack interpretability, hindering trust and debugging. Furthermore, achieving precise trajectory tracking, especially in complex scenarios, remains a significant challenge. This paper aims to bridge this gap by developing an interpretable, data-driven model that accurately captures vehicle dynamics and facilitates robust trajectory tracking.

Methods

The core of this work is the Deep Direct Koopman (DDK) approach. DDK builds upon Koopman operator theory, which offers a way to linearize nonlinear dynamical systems in an infinite-dimensional space. The key innovations of DDK are:

  • Direct Koopman Eigenvalue Learning: Instead of learning a complex mapping, DDK directly learns the Koopman eigenvalues and system matrices as trainable tensors. This simplifies the identification process.
  • Interpretable Latent Space: DDK concatenates the original vehicle state (e.g., position, velocity) with features extracted by an encoder. This ensures that the first part of the learned latent state directly corresponds to the physical vehicle state, providing interpretability.
  • Structured Transition Matrix: The learned Koopman eigenvalues are used to construct a block-diagonal state transition matrix, which is crucial for representing the linearized dynamics.
  • DDK-MPC Controller: The identified linear dynamics are then used to design a Model Predictive Control (MPC) controller, termed DDK-MPC. This controller leverages the learned linear model to predict future states and optimize control inputs for trajectory tracking.

Datasets

  • CarSim Simulation Data: Approximately 60km of data collected from a real electric SUV was used for training and validation. This dataset includes various driving scenarios like straight roads, U-turns, roundabouts, and lane changes.
  • CarSim 2019 Environment: Used to generate high-fidelity simulation data for training, validation, and testing of the DDK-MPC controller.

Key Insights and Ideas

  • Linearization of Nonlinear Dynamics: The Koopman operator provides a powerful theoretical framework for linearizing complex nonlinear systems, which DDK effectively harnesses.
  • Interpretability through State Concatenation: By directly including the physical state in the latent representation, DDK offers a degree of interpretability that is often missing in purely data-driven models. This allows for understanding how different physical states contribute to the learned dynamics.
  • Direct Learning of Koopman Parameters: Learning the Koopman eigenvalues and system matrices directly, rather than through intermediate steps, simplifies the model and potentially improves performance.
  • DDK-MPC for Robust Tracking: The combination of an interpretable Koopman-based model with MPC proves effective for precise point-to-point (P2P) trajectory tracking, even in challenging scenarios.

Key Results

  • Accurate Dynamics Modeling: DDK demonstrated superior performance in approximating vehicle dynamics compared to other deep learning-based Koopman methods like DeepEDMD and LRAN, as evidenced by lower Root Mean Square Errors (RMSE) in prediction tasks.
  • Effective Trajectory Tracking: The DDK-MPC controller achieved excellent P2P trajectory tracking performance, successfully following reference trajectories in simulations within CarSim. It outperformed other MPC methods, including LTV-MPC and PurePursuit, particularly in complex maneuvers.
  • Real-World Validation: Experiments with a real electric SUV showed that DDK could accurately model vehicle dynamics, with prediction results demonstrating its effectiveness even with slightly challenging longitudinal velocity predictions.

Limitations and Future Work

  • Constraint Handling: The paper notes that constraints on states were not explicitly added to the model to avoid unsolvable situations during simulation. This could lead to large offsets if the identified model is imprecise.
  • Computational Cost: While DDK aims for real-time performance, the computational cost of training and inference, especially for complex models, can still be a factor.
  • Future Work: The authors propose extending DDK to address parameter tuning challenges in MPC and applying DDK-MPC to more complex motion planning scenarios. Further research will also explore learning-based methods for trajectory tracking to address the limitations of traditional MPC.

Summary updated

Abstract

Autonomous driving has attracted lots of attention in recent years. For some tasks, e.g., trajectory prediction, motion planning, and trajectory tracking, an accurate vehicle model can reduce the difficulty of these tasks and improve task completion performance. Prior works focused on parameter estimation of physical models or modeling nonlinear dynamics using neural networks. Still, these methods rely on internal parameters of vehicles or are not friendly for control due to the strong nonlinearity of models. This paper proposes a data-driven method to approximate vehicle dynamics based on the Koopman operator. The resulting model is an interpretable linear time-invariant model, facilitating controller design and solving related optimization problems. In the proposed approach, the state transition matrix is constructed based on the learned Koopman eigenvalues, while the input matrix is trained as a tensor. Based on the resulting model, a linear model predictive controller is designed to implement coupled longitudinal and lateral trajectory tracking. Simulations and experiments, including vehicle dynamics modeling and coupled longitudinal and lateral trajectory tracking, are performed in a high-fidelity CarSim environment and a real vehicle platform. An oil-driven D-Class SUV is selected in the simulation, while a real electric SUV is utilized in the experiment. Simulation and experiment results illustrate that the model of the nonlinear vehicle dynamics can be identified effectively via the proposed method, and high-quality trajectory tracking performance can be obtained with the resulting model.
BibTeX
@inproceedings{xiao2023ddk,
  author = {Yongqian Xiao and Xinglong Zhang and Xin Xu and Yang Lu and Junxiang Li},
  title = {{DDK: A Deep Koopman Approach for Longitudinal and Lateral Control of Autonomous Ground Vehicles}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10161104},
}
Figure