Deep Model Reference Control.

Scott Bout, Dennis McCann, Girish Chowdhary2022 · DOI

TL;DR

Deep Model Predictive Control (Deep MPC) combines the rigorous constraint-handling of traditional control theory with the flexible learning power of deep neural networks. By using a dual-rate learning architecture, this approach allows autonomous systems to adapt to sudden environmental changes and unmodeled dynamics in real-time without compromising safety.

What Problem Does This Paper Address?

Traditional Model Predictive Control (MPC) is a workhorse in engineering because it can optimize performance while strictly adhering to physical constraints (like speed limits or battery levels). However, MPC relies on a mathematical model of the "plant" (the system being controlled). In the real world, these models are never perfect.

Environmental uncertainties—such as sudden wind gusts for a drone or mechanical wear and tear—create a gap between how the controller thinks the system will behave and how it actually behaves. Basic model-based controllers often exhibit unexpected or unsafe behaviors when faced with these "unmodeled dynamics." The authors seek to bridge this gap by using Deep Neural Networks (DNNs) to learn and mitigate these uncertainties on the fly.

Methods

The authors propose a hybrid control framework that splits the control signal into two parts:

  1. The Nominal MPC ($u^m$): Handles the known physics and ensures the system respects its physical and trajectory constraints.
  2. The Adaptive Element ($u^a$): A Deep Neural Network that learns the "unknown" part of the dynamics ($h(x_t)$) and provides a corrective signal to cancel out disturbances.

Key Innovations: * Dual-Rate Learning: The system employs two networks operating at different speeds. A "Fast Learning" outer loop operates at 100 Hz for immediate adaptation, while a "Slower Feature Learning" loop operates at 10 Hz to update the network's underlying understanding of the environment. * Experience Replay with SVD: To prevent the network from "forgetting" old information or learning from redundant data, they use a replay buffer. They specifically use Singular Value Decomposition (SVD) maximization to ensure the buffer only adds data that increases the "richness" or diversity of the training set. * Deep vs. Shallow: Unlike previous adaptive controllers that used shallow networks, this approach utilizes deep architectures (2+ hidden layers) to capture more complex, nonlinear uncertainties.

Datasets

The paper does not use standard static datasets. Instead, it evaluates the algorithm using:

  • Simulated Wing-Rock Dynamics: A mathematical model of a one-degree-of-freedom system (representing aircraft roll) used to test how the controller handles abrupt changes in uncertainty.
  • Future Benchmarks: The authors mention upcoming physical experimentation using the Crazyflie 2.0 Quadcopter platform.

Key Insights and Ideas

  • Separation of Concerns: By treating the DNN as an additive correction to a standard MPC, the system retains the theoretical safety guarantees of MPC while gaining the flexibility of AI.
  • The Importance of "Rich" Data: The use of SVD for data selection is a clever way to solve the "stability vs. plasticity" dilemma in online learning. It ensures the controller learns from meaningful errors rather than just noise.
  • Deep Learning for Control: The results suggest that "going deep" isn't just for computer vision; deeper networks are significantly better at approximating the complex, state-dependent uncertainties found in aerodynamics than shallow networks.

Key Results

In simulations of wing-rock dynamics (stabilizing an aircraft's roll angle and rate):

  • Deep MPC achieved near-perfect stabilization, bringing the roll angle to zero quickly and smoothly even after abrupt changes in uncertainty.
  • Tube MPC (a standard robust control method) exhibited significant oscillations and failed to settle at zero as effectively.
  • Shallow MPC performed better than Tube MPC but lacked the precision and rapid convergence of the Deep MPC approach.

Limitations and Future Work

The primary limitation is that the current results are based on simulations. While the math is promising, real-world hardware introduces latency and sensor noise that simulations often overlook.

Future Work includes: * Physical Testing: Moving the algorithm onto a Crazyflie 2.0 quadcopter. * Stress Testing: Evaluating the system against specific hardware failures (propeller damage), environmental factors (wind bias), and payload changes (hanging weights). * Architecture Exploration: Experimenting with different machine learning architectures to see which are most efficient for real-time adaptive control.

Summary updated

Abstract

Abstract not provided.
BibTeX
@misc{bout2022deep,
  author = {Scott Bout and Dennis McCann and Girish Chowdhary},
  title = {{Deep Model Reference Control.}},
  year = {2022},
  doi = {10.2172/2004692},
}

Loading related content...

Figure