TL;DR
This paper introduces DISKO, an algorithm for learning stable Koopman operators from data. By enforcing stability, DISKO improves long-term prediction accuracy and enables the construction of Lyapunov functions for verifying stabilizing controllers, outperforming standard methods in simulations and experiments.
What Problem Does This Paper Address?
Many real-world systems, like robots and their environments, are inherently nonlinear and often difficult to model accurately. Data-driven approaches, particularly those using Koopman operators, offer a way to represent these nonlinear dynamics linearly. However, standard methods for learning Koopman operators can result in unstable models, even for stable systems. This instability leads to exponentially diverging predictions, making them impractical for long-term control and analysis. The paper addresses this by proposing a method to learn stable Koopman operators, ensuring that predictions remain accurate and reliable over extended time horizons and enabling the development of robust control strategies.
Methods
The core contribution is the Data-driven Identification of Stable Koopman Operators (DISKO). This method uses an algorithm (specifically, a "SOC" algorithm based on [1]) that finds the nearest stable matrix solution to a least-squares reconstruction error. This is a departure from standard approaches that might simply project an unstable solution onto the set of stable matrices, potentially sacrificing accuracy.
Key methodological contributions include:
- Prediction Error Formula: Deriving a formula for the prediction error of Koopman representations over arbitrary time steps, demonstrating how stability constraints improve long-term accuracy.
- Basis Function Conditions: Establishing formal conditions for selecting basis functions that are consistent with the stability properties of the underlying nonlinear system.
- Lyapunov Function Construction: Developing a method to construct Lyapunov functions for nonlinear systems using stable data-driven Koopman operators, which can then be used to verify stabilizing controllers.
Datasets
The paper uses data generated from simulations of:
- A pendulum system.
- A quadrotor system.
- A pusher-slider system (experimental data).
The data for training and evaluation is collected by simulating the dynamics of these systems with various initial conditions and control inputs.
Key Insights and Ideas
- Stability is Crucial for Long-Term Prediction: Unstable Koopman operators, even if they fit the training data well, can lead to wildly inaccurate predictions over time due to the exponential amplification of modeling errors.
- Stable Models Improve Data Efficiency: DISKO's stability constraints make the learned models more robust to limited training data, a common issue in real-world applications.
- Basis Functions Matter for Stability: The choice of basis functions is not arbitrary; they must be consistent with the stability properties of the original nonlinear system to ensure a stable Koopman representation.
- Stable Koopman Operators Enable Control Verification: The ability to construct Lyapunov functions from stable Koopman operators provides a powerful tool for formally verifying the stability of control policies.
Key Results
- Improved Prediction Accuracy: DISKO consistently outperforms unconstrained least-squares Koopman models in prediction accuracy for pendulum and quadrotor systems, especially in low-data regimes and for longer prediction horizons. For example, for the undamped pendulum, the average prediction error was 0.52 radians for DISKO versus 1.03 radians for the least-squares model.
- Robust Control Performance: In experiments with a pusher-slider system, DISKO-based LQR control demonstrated more robust performance, keeping the block closer to the target end goal compared to control derived from an unstable Koopman model.
- Lyapunov Function Validation: The paper successfully demonstrates the construction and validation of candidate control-Lyapunov functions for controlled systems (pendulum and cart-pendulum), verifying stabilizing control trajectories.
Limitations and Future Work
- Basis Function Selection: While the paper provides conditions for choosing basis functions, finding optimal basis functions that satisfy these conditions and improve model accuracy remains an active research area.
- Equilibrium Points: For systems with multiple equilibrium points, ensuring that the Koopman model shares the same equilibrium points as the original nonlinear system requires further investigation.
- Adaptive Stability: The paper suggests exploring adaptive algorithms that can learn the optimal level of stability or instability based on data.
- Koopman Invariant Subspaces: Investigating the use of Koopman invariant subspaces for improving model accuracy and control is identified as a future direction.
- Online Learning: While the presented SOC algorithm is memory-efficient, further work on online learning and real-time adaptation of stable Koopman models is suggested.
Summary updated