Learning Dissipative Neural Dynamical Systems

Yuezhu Xu, S. Sivaranjani

Introduction

In modern control theory and robotics, the integration of Deep Learning with dynamical systems has led to the rise of Neural Ordinary Differential Equations (Neural ODEs). While these models excel at capturing complex, nonlinear dynamics from data, they often function as "black boxes" lacking the inherent physical guarantees required for safety-critical applications. Specifically, standard Neural ODEs do not naturally satisfy dissipativity—a fundamental property that generalizes energy storage and dissipation. A dissipative system is inherently stable and robust to perturbations, making it a reliable candidate for controller design. This paper introduces a novel framework to bridge the gap between high-capacity neural modeling and rigorous physical consistency.

The Challenge of Constrained Learning

Training a neural network to respect physical laws is computationally demanding. These constraints are typically expressed as complex matrix inequalities that must be satisfied throughout the training process. In the context of Neural ODEs, forcing the model to remain dissipative often leads to optimization instability or a significant loss in predictive accuracy. Consequently, even when a neural network is trained on data from a known dissipative physical system, the resulting model often fails to inherit those dissipative properties, rendering it potentially unstable when used in a closed-loop control setting.

The "Train-Then-Perturb" Framework

The authors propose a three-stage algorithm designed to decouple the enforcement of physical constraints from the initial fitting of the data. This approach allows for the use of standard deep learning tools while ensuring the final model is mathematically guaranteed to be dissipative.

1. Baseline Training

The process begins with training a standard, unconstrained Neural ODE. The goal of this stage is purely to minimize the error between the model's predicted trajectories and the observed system data. At this point, the model is a high-fidelity "black box" with no guarantees regarding its stability or dissipativity.

2. Weight Perturbation for Dissipativity

To enforce dissipativity, the authors leverage the property of slope-restrictedness in common activation functions (such as ReLU or Tanh). By bounding the derivatives of these functions, the requirement for dissipativity can be translated into a Linear Matrix Inequality (LMI).

The framework then solves a convex optimization problem to find the smallest possible perturbation to the network's weights ($W$) that satisfies this LMI. This step ensures that the model is provably dissipative while staying as close as possible to the high-performing baseline model.

3. Bias Adjustment for Accuracy

A key insight of this research is that dissipativity in this specific neural architecture depends solely on the weight matrices, not the bias vectors ($b$). Because the weight perturbation in the previous step inevitably degrades the model's fit, the authors perform a final round of training. In this stage, the dissipative weights are "frozen," and only the biases are optimized to recover the model's original accuracy. This "Weight-Bias Split" allows the model to satisfy physical laws (via weights) and empirical data (via biases) sequentially.

Experimental Evaluation: The Duffing Oscillator

The framework was validated using a second-order Duffing oscillator, a classic benchmark for nonlinear dynamics. The authors generated synthetic trajectories with added Gaussian noise to simulate real-world measurement conditions.

Key Results

  • Mathematical Guarantees: While the baseline model failed passivity tests, the perturbed model was mathematically proven to be dissipative.
  • Minimal Distortion: The optimization successfully found a dissipative solution with a weight perturbation 2-norm of only 1.4443 for a 128-dimensional vector, indicating that the physical constraints did not require a radical departure from the data-driven model.
  • Trajectory Fidelity: Following the bias adjustment phase, the dissipative model's predicted trajectories almost perfectly overlapped with the ground truth, effectively eliminating the drift caused by the weight perturbation.

Conclusion and Future Directions

This research provides a practical pathway for deploying Neural ODEs in environments where stability is non-negotiable. By treating physics and accuracy as separate optimization targets, the authors demonstrate that we do not have to sacrifice the expressive power of deep learning to achieve physical consistency.

However, challenges remain regarding the scalability of solving LMIs as network depth and width increase. Future work may explore compositional approaches to handle larger architectures. Additionally, extending this framework to systems with partial state observations and integrating these dissipative models directly into closed-loop controller synthesis represent promising frontiers for robust, AI-driven control.

Summary updated

Abstract

Consider an unknown nonlinear dynamical system that is known to be dissipative. The objective of this letter is to learn a neural dynamical model that approximates this system, while preserving the dissipativity property in the model. In general, imposing dissipativity constraints during neural network training is a hard problem for which no known techniques exist. In this letter, we address the problem of learning a dissipative neural dynamical system model in two stages. First, we learn an unconstrained neural dynamical model that closely approximates the system dynamics. Next, we derive sufficient conditions to perturb the weights of the neural dynamical model to ensure dissipativity, followed by perturbation of the biases to retain the fit of the model to the trajectories of the nonlinear system. We show that these two perturbation problems can be solved independently to obtain a neural dynamical model that is guaranteed to be dissipative while closely approximating the nonlinear system.
BibTeX
@article{xu2023learning,
  author = {Yuezhu Xu and S. Sivaranjani},
  title = {{Learning Dissipative Neural Dynamical Systems}},
  journal = {IEEE Control. Syst. Lett.},
  year = {2023},
  doi = {10.1109/lcsys.2023.3337851},
}
Figure