Abstract
Traditional stability analysis for nonlinear systems often relies on Lyapunov-based methods that struggle with high dimensionality and non-analytical components like neural network controllers or raw image inputs. This paper proposes repurposing Projected Gradient Descent (PGD) attacks—originally designed for adversarial machine learning—to estimate the Region of Attraction (ROA) of complex control systems. By treating the search for unstable initial states as an adversarial optimization problem, the proposed method achieves linear scalability and enables the analysis of perception-based controllers operating directly on camera pixels.
Introduction
In control theory, the Region of Attraction (ROA) defines the set of initial states from which a system is guaranteed to converge to an equilibrium point. Determining the ROA is critical for safety-critical applications. However, existing rigorous techniques, such as Sum-of-Squares (SOS) or Linear Matrix Inequalities (LMI), face two significant hurdles:
- Scalability: Computational complexity grows exponentially with system dimensions, often failing beyond a few dozen variables.
- The Perception Gap: Modern controllers often use Deep Reinforcement Learning (RL) to map raw sensor data (e.g., RGB images) directly to actions. These "perception-based" systems lack the analytical models required by traditional mathematical proofs.
Methodology: Adversarial Stability Analysis
The authors frame ROA estimation as a constrained maximization problem. Instead of proving stability, they attempt to "attack" the system by finding the worst-case initial condition within a candidate safe zone that maximizes the system's distance from equilibrium after a time horizon $T$.
Model-Based and Model-Free PGD
Two variants of the PGD-based tool are introduced:
- Model-Based PGD: When system dynamics are known, backpropagation is used to calculate gradients, allowing the starting point to be iteratively adjusted toward instability.
- Model-Free PGD: For systems where dynamics are unknown or involve complex perception pipelines, the method uses finite-difference estimates of the gradient. This treats the system as a black box, requiring only a simulator to evaluate trajectories.
A key innovation is a closed-form update rule that improves efficiency by focusing the search on the boundary of the candidate safe zone, where instabilities are most likely to occur.
Perception-Based Control
A major contribution of this work is the ability to analyze systems where the controller processes high-dimensional visual input.
In these scenarios, the controller (NN Policy $K$) receives a history of observations derived from an image buffer of RGB images $y_t$. Traditional Lyapunov functions cannot be easily constructed for such high-dimensional, non-linear mappings. The PGD approach bypasses this by operating on the input space of the simulator, allowing researchers to estimate the stability of systems that rely on raw pixels.
Experimental Evaluation
The method was tested on several benchmarks to demonstrate its versatility:
- High-Dimensional Scaling: A synthetic cubic system was used to demonstrate that PGD scales linearly. The authors successfully analyzed a 1,000-dimensional system in under four hours on a standard laptop—a task impossible for SOS methods.
- Deepmind Control Suite: Inverted pendulum and cart-pole systems were evaluated. For the perception-based task, the controller received $84 \times 84$ RGB images.
- Comparison with LMI: On lower-dimensional systems, PGD provided larger, less conservative ROA estimates than traditional LMI methods, which often underestimate the true safe zone.
Discussion and Conclusion
While the PGD-based approach offers unprecedented scalability and flexibility, it provides an empirical approximation rather than a formal mathematical proof. As a local search method, it may miss small pockets of instability. However, its ability to handle 1,000-dimensional states and raw pixel inputs makes it a powerful tool for modern robotics. Future work aims to extend this framework to Input-Output Gain Analysis ($H_\infty$ control) to assess robustness against continuous external disturbances.
Summary updated