Learning Control-Oriented Dynamical Structure from Data

Spencer M. Richards, Jean-Jacques Slotine, Navid Azizan, Marco Pavone

Abstract

This work presents a novel framework for jointly learning nonlinear dynamics and their control-oriented structure via state-dependent coefficient (SDC) factorizations. The resulting SD-LQR approach bridges the gap between system identification and control design, offering superior data efficiency and robustness compared to methods that learn stability certificates from scratch.

Introduction

In autonomous systems, a significant disconnect often exists between system identification and control synthesis. While deep learning can model complex nonlinear dynamics $\dot{x} = f(x) + B(x)u$, translating these models into effective controllers remains challenging. Standard Linear Quadratic Regulators (LQR) are restricted to local equilibria, while modern "certified" methods—which attempt to learn controllers and Lyapunov functions simultaneously—are often data-intensive and fragile. This paper proposes learning a specific mathematical structure that simplifies nonlinear control without the brittleness of explicit certificates.

Methodology

The core of the proposed approach is the use of State-Dependent Coefficient (SDC) factorizations. Any smooth nonlinear system's error dynamics can be represented in an exact, linear-like form: $\dot{e} = A(x, \bar{x}, u)e + B(x)v$. Unlike local linearizations, this factorization holds globally.

By representing the dynamics ($f, B$) and the SDC factors ($A$) with neural networks, the authors employ a semi-supervised loss function to ensure the learned $A$ matrix accurately factorizes the learned dynamics $f$. This structure allows for solving a State-Dependent Riccati Equation (SDRE) online, effectively performing LQR at every timestep using the true nonlinear structure rather than a local approximation.

Experimental Results

The method was evaluated on two benchmarks: a 6D Planar Spacecraft and a highly coupled, underactuated Planar Vertical-Take-Off-and-Landing (PVTOL) system.

Trajectory tracking performance on the PVTOL system across varying dataset sizes (N). The proposed SD-LQR (green) maintains high accuracy even with minimal data (N=50), while competing methods like CCM (orange) struggle.

A key finding is that embedding SDC structure acts as a powerful regularizer, preventing the model from learning physically implausible behaviors. In the PVTOL "double loop-the-loop" task, SD-LQR achieved stable tracking with as few as 50-100 data points, whereas certificate-based methods like Control Contraction Metrics (CCM) required significantly more data to avoid failure.

Comparative error decay and robustness for Spacecraft and PVTOL benchmarks. SD-LQR (green) shows consistent performance and lower variance compared to LQR and CCM across different training sizes.

Quantitatively, SD-LQR consistently maintained tracking errors an order of magnitude lower than naive LQR applied to learned models. Furthermore, the method exhibited high robustness, with tighter performance distributions (interquartile ranges) across random seeds compared to the high variance seen in certificate-based approaches.

Conclusion

SD-LQR demonstrates that learning control-oriented structure is a viable alternative to learning explicit stability proofs. While it currently provides local rather than global stability guarantees and requires online Riccati equation solving, its data efficiency and robustness make it highly suitable for complex robotic tasks. Future work includes developing architectures that are intrinsically stabilizable by construction, further reducing the need for complex penalty terms during training.

Summary updated

Abstract

Even for known nonlinear dynamical systems, feedback controller synthesis is a difficult problem that often requires leveraging the particular structure of the dynamics to induce a stable closed-loop system. For general nonlinear models, including those fit to data, there may not be enough known structure to reliably synthesize a stabilizing feedback controller. In this paper, we discuss a state-dependent nonlinear tracking controller formulation based on a state-dependent Riccati equation for general nonlinear control-affine systems. This formulation depends on a nonlinear factorization of the system of vector fields defining the control-affine dynamics, which always exists under mild smoothness assumptions. We propose a method for learning this factorization from a finite set of data. On a variety of simulated nonlinear dynamical systems, we empirically demonstrate the efficacy of learned versions of this controller in stable trajectory tracking. Alongside our learning method, we evaluate recent ideas in jointly learning a controller and stabilizability certificate for known dynamical systems; we show experimentally that such methods can be frail in comparison.
BibTeX
@inproceedings{richards2023learning,
  author = {Spencer M. Richards and Jean-Jacques Slotine and Navid Azizan and Marco Pavone},
  title = {{Learning Control-Oriented Dynamical Structure from Data}},
  booktitle = {International Conference on Machine Learning},
  year = {2023},
  url = {https://openreview.net/forum?id=oke1MUPK2l},
}
Figure