Revisiting PGD Attacks for Stability Analysis of Large-Scale Nonlinear Systems and Perception-Based Control

Aaron Havens, Darioush Keivan, Peter Seiler, Geir E. Dullerud, Bin HuarXiv 2022 · OpenReview

Abstract

Traditional stability analysis for nonlinear systems often relies on Lyapunov-based methods that struggle with high dimensionality and non-analytical components like neural network controllers or raw image inputs. This paper proposes repurposing Projected Gradient Descent (PGD) attacks—originally designed for adversarial machine learning—to estimate the Region of Attraction (ROA) of complex control systems. By treating the search for unstable initial states as an adversarial optimization problem, the proposed method achieves linear scalability and enables the analysis of perception-based controllers operating directly on camera pixels.

Introduction

In control theory, the Region of Attraction (ROA) defines the set of initial states from which a system is guaranteed to converge to an equilibrium point. Determining the ROA is critical for safety-critical applications. However, existing rigorous techniques, such as Sum-of-Squares (SOS) or Linear Matrix Inequalities (LMI), face two significant hurdles:

  1. Scalability: Computational complexity grows exponentially with system dimensions, often failing beyond a few dozen variables.
  2. The Perception Gap: Modern controllers often use Deep Reinforcement Learning (RL) to map raw sensor data (e.g., RGB images) directly to actions. These "perception-based" systems lack the analytical models required by traditional mathematical proofs.

Methodology: Adversarial Stability Analysis

The authors frame ROA estimation as a constrained maximization problem. Instead of proving stability, they attempt to "attack" the system by finding the worst-case initial condition within a candidate safe zone that maximizes the system's distance from equilibrium after a time horizon $T$.

Model-Based and Model-Free PGD

Two variants of the PGD-based tool are introduced:

  • Model-Based PGD: When system dynamics are known, backpropagation is used to calculate gradients, allowing the starting point to be iteratively adjusted toward instability.
  • Model-Free PGD: For systems where dynamics are unknown or involve complex perception pipelines, the method uses finite-difference estimates of the gradient. This treats the system as a black box, requiring only a simulator to evaluate trajectories.

A key innovation is a closed-form update rule that improves efficiency by focusing the search on the boundary of the candidate safe zone, where instabilities are most likely to occur.

Perception-Based Control

A major contribution of this work is the ability to analyze systems where the controller processes high-dimensional visual input.

Perception-based control loop architecture.

In these scenarios, the controller (NN Policy $K$) receives a history of observations derived from an image buffer of RGB images $y_t$. Traditional Lyapunov functions cannot be easily constructed for such high-dimensional, non-linear mappings. The PGD approach bypasses this by operating on the input space of the simulator, allowing researchers to estimate the stability of systems that rely on raw pixels.

Experimental Evaluation

The method was tested on several benchmarks to demonstrate its versatility:

  • High-Dimensional Scaling: A synthetic cubic system was used to demonstrate that PGD scales linearly. The authors successfully analyzed a 1,000-dimensional system in under four hours on a standard laptop—a task impossible for SOS methods.
  • Deepmind Control Suite: Inverted pendulum and cart-pole systems were evaluated. For the perception-based task, the controller received $84 \times 84$ RGB images.
  • Comparison with LMI: On lower-dimensional systems, PGD provided larger, less conservative ROA estimates than traditional LMI methods, which often underestimate the true safe zone.

Discussion and Conclusion

While the PGD-based approach offers unprecedented scalability and flexibility, it provides an empirical approximation rather than a formal mathematical proof. As a local search method, it may miss small pockets of instability. However, its ability to handle 1,000-dimensional states and raw pixel inputs makes it a powerful tool for modern robotics. Future work aims to extend this framework to Input-Output Gain Analysis ($H_\infty$ control) to assess robustness against continuous external disturbances.

Summary updated

Abstract

Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this paper, we tailor the projected gradient descent (PGD) attack method developed in the adversarial learning community as a general-purpose ROA analysis tool for large-scale nonlinear systems and end-to-end perception-based control. We show that the ROA analysis can be approximated as a constrained maximization problem whose goal is to find the worst-case initial condition which shifts the terminal state the most. Then we present two PGD-based iterative methods which can be used to solve the resultant constrained maximization problem. Our analysis is not based on Lyapunov theory, and hence requires minimum information of the problem structures. In the model-based setting, we show that the PGD updates can be efficiently performed using back-propagation. In the model-free setting (which is more relevant to ROA analysis of perception-based control), we propose a finite-difference PGD estimate which is general and only requires a black-box simulator for generating the trajectories of the closed-loop system given any initial state. We demonstrate the scalability and generality of our analysis tool on several numerical examples with large-scale NN policies and high-dimensional image observations. We believe that our proposed analysis serves as a meaningful initial step toward further understanding of closed-loop stability of large-scale nonlinear systems and perception-based control.
BibTeX
@misc{havens2022revisiting,
  author = {Aaron Havens and Darioush Keivan and Peter Seiler and Geir E. Dullerud and Bin Hu},
  title = {{Revisiting PGD Attacks for Stability Analysis of Large-Scale Nonlinear Systems and Perception-Based Control}},
  year = {2022},
  eprint = {2201.00801},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2201.00801},
}
Figure