DeSKO: Stability-Assured Robust Control with a Deep Stochastic Koopman Operator

Minghao Han, Jacob Euler-Rolle, Robert K. KatzschmannICLR 2022 · OpenReview

Introduction

Researchers have unveiled DeSKO (Deep Stochastic Koopman Operator), a novel framework that tames complex, noisy nonlinear systems by mapping them into a high-dimensional linear space. Unlike previous methods, DeSKO models uncertainty as a probability distribution and provides rigorous mathematical guarantees for stability. This breakthrough allows robots to resist external disturbances up to five times their own control strength, paving the way for safer and more robust real-world automation.

The Challenge of Nonlinear Control

Controlling nonlinear systems—like a floppy soft robotic arm or a multi-legged walker—is notoriously difficult. Engineers often rely on Koopman operator theory, which mathematically "lifts" these messy nonlinear dynamics into a higher-dimensional space where the system behaves linearly. Linear systems are highly desirable because decades of efficient, well-understood control tools exist for them.

However, standard Koopman models are deterministic; they assume a perfect, noise-free world. In reality, sensors are jittery (observation noise) and environments are unpredictable (process noise). When a deterministic model meets a noisy world, it often fails or becomes unstable. Furthermore, while deep learning can model these complex systems, neural networks act as "black boxes" that rarely offer the stability guarantees required for safety-critical robotics.

The DeSKO Methodology

The authors introduce DeSKO to combine deep learning’s representational power with the structural rigor of Koopman theory.

DeSKO Architecture
  1. Deep Stochastic Mapping: Instead of mapping a state to a single point in the "lifted" space, DeSKO uses a probabilistic neural network to infer a distribution (mean and variance) of observables.
  2. Linear Propagation: It learns Koopman matrices ($A$ and $B$) that propagate this entire distribution forward in time linearly. This maintains the "linear-is-easier" advantage while explicitly accounting for uncertainty.
  3. Entropy Constraint: To prevent the model from becoming "overconfident" in regions with little data, an entropy constraint is added during training. This forces the model to maintain a minimum level of uncertainty, keeping it robust to unseen disturbances.
  4. Robust MPC with LQR Feedback: The controller uses Model Predictive Control (MPC) to plan a "nominal" path based on the mean of the distribution. It then uses a Linear Quadratic Regulator (LQR) as a feedback mechanism to constantly nudge the actual, noisy system back toward that nominal path.
  5. Stability Proofs: Crucially, the paper provides mathematical proofs showing that even with approximation errors and noise, the system remains "uniformly ultimately bounded"—meaning it is mathematically guaranteed not to spiral out of control.

Key Insights

  • Linearity is for Distributions, too: The most elegant insight is that the Koopman operator can propagate a probability distribution just as easily as a single point, allowing for "uncertainty-aware" linear control.
  • The "Overconfidence" Trap: Without the entropy constraint, models tend to ignore noise in areas where they have seen limited data. By forcing the model to "stay humble" (maintain entropy), the resulting controller becomes significantly more robust.
  • Bridging Theory and Practice: While many papers focus on either pure theory (stability proofs) or pure performance (deep RL), DeSKO manages to achieve state-of-the-art performance on physical robots while maintaining a theoretical safety net.

Key Results

The framework was evaluated across eight diverse environments, ranging from classic benchmarks (CartPole) to high-dimensional biological models (Gene Regulatory Networks) and robotic systems (HalfCheetah, Soft Robotic Arms).

Performance across benchmarks
  • Superior Robustness: In the CartPole test, DeSKO resisted external impulsive forces up to 5x the magnitude of the maximum control input, whereas baseline methods (like standard Deep Koopman Operators or Soft Actor-Critic) failed much earlier.
  • Better Scaling: DeSKO outperformed traditional Koopman methods on high-dimensional tasks like the HalfCheetah and the soft robotic arm, proving it can handle systems with many degrees of freedom.
  • Real-World Success: The team successfully deployed DeSKO on a physical soft robotic arm made of silicon to perform dynamic trajectory tracking (drawing letters in the air). It significantly outperformed deterministic baselines in tracking accuracy.
  • Lower Prediction Error: Across all noisy benchmarks, DeSKO maintained lower cumulative prediction errors over long time horizons compared to deterministic models.

Limitations and Future Work

While DeSKO is a major step forward, the authors note a few areas for future exploration:

  • Training Convergence: While the controller is proven stable, a formal convergence proof for the training algorithm itself (the stochastic gradient descent part) remains an open challenge.
  • Nonlinear Inputs: The current model assumes control inputs enter the system linearly. Future work could explore "control-affine" or fully nonlinear Koopman representations to handle even more complex actuators.
  • State Constraints: The framework doesn't yet explicitly handle hard constraints (e.g., "don't hit this wall"), which is a common requirement in industrial robotics.

Summary updated

Abstract

The Koopman operator theory linearly describes nonlinear dynamical systems in a high-dimensional functional space and it allows to apply linear control methods to highly nonlinear systems. However, the Koopman operator does not account for any uncertainty in dynamical systems, causing it to perform poorly in real-world applications. Therefore, we propose a deep stochastic Koopman operator (DeSKO) model in a robust learning control framework to guarantee stability of nonlinear stochastic systems. The DeSKO model captures a dynamical system's uncertainty by inferring a distribution of observables. We use the inferred distribution to design a robust, stabilizing closed-loop controller for a dynamical system. Modeling and control experiments on several advanced control benchmarks show that our framework is more robust and scalable than state-of-the-art deep Koopman operators and reinforcement learning methods. Tested control benchmarks include a soft robotic arm, a legged robot, and a biological gene regulatory network. We also demonstrate that this robust control method resists previously unseen uncertainties, such as external disturbances, with a magnitude of up to five times the maximum control input. Our approach opens up new possibilities in learning control for high-dimensional nonlinear systems while robustly managing internal or external uncertainty.
BibTeX
@inproceedings{han2022desko,
  author = {Minghao Han and Jacob Euler-Rolle and Robert K. Katzschmann},
  title = {{DeSKO: Stability-Assured Robust Control with a Deep Stochastic Koopman Operator}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=hniLRD_XCA},
}
Figure