Nonlinear Joint Spectral Radius

Piero Deidda, Nicola Guglielmi, Francesco TudiscoarXiv 2025 · DOI · arXiv

TL;DR

This paper extends the Joint Spectral Radius (JSR)—a fundamental tool for analyzing the stability of linear switched systems—to a broad class of nonlinear systems. By focusing on "sub-homogeneous" and "order-preserving" maps, the authors provide a rigorous framework to guarantee stability and robustness in complex models like deep neural networks.

What Problem Does This Paper Address?

In many real-world systems, the rules of evolution change over time. Think of a discrete-time system $x_{k+1} = f_{\sigma(k)}(x_k)$, where at each step $k$, we pick a function $f$ from a family $\mathcal{F}$ based on a switching rule $\sigma$.

If these functions are linear (matrices), we use the Joint Spectral Radius (JSR) to determine if the system is stable. The JSR tells us the maximum possible growth rate across all possible switching sequences. If JSR < 1, the system eventually settles to zero.

However, most modern systems—especially Deep Neural Networks (DNNs)—are nonlinear. While we can look at local stability using Jacobians (derivatives), these often fail to capture the global behavior of the system. There has been a significant theoretical gap in defining a "Global JSR" for nonlinear systems that can handle the "worst-case" switching scenarios common in machine learning and control theory.

Methods

The authors bridge this gap by focusing on maps that act on cones (like the non-negative orthant $\mathbb{R}^N_+$) and satisfy two key properties:

  1. Sub-homogeneity: $f(\lambda x) \leq \lambda f(x)$ for $\lambda \geq 1$. (Scaling the input scales the output by at most the same amount).
  2. Order-preservation: $x \leq y \implies f(x) \leq f(y)$. (The map respects the natural ordering of the cone).

Key Technical Contributions:

  • Asymptotic Scaling: They show that a complex nonlinear (sub-homogeneous) system can be "sandwiched" between two simpler homogeneous systems: one representing behavior near zero ($f_0$) and one near infinity ($f_\infty$).
  • Dual Formulation: They extend the classical Rota-Strang theorem to the nonlinear setting, proving that the JSR can be characterized using monotone prenorms.
  • The Polytopal Algorithm: They adapt a classical linear algorithm to the nonlinear world. This algorithm iteratively builds a "unit ball" (a set of points) to approximate the JSR and provides conditions under which it converges in finite time.

Datasets

As a theoretical mathematics and stability analysis paper, it does not use standard machine learning benchmarks (like ImageNet or CIFAR-10). Instead, the authors use:

  • Synthetic Numerical Examples: Specifically designed $2 \times 2$ nonlinear systems (e.g., Example 6.6 and 6.7) to demonstrate the convergence and accuracy of their proposed Polytopal Algorithm.
  • Theoretical Models of Neural Layers: They model standard layers (Linear + ReLU/Sigmoid/Softplus) to show how their framework applies to AI architectures.

Key Insights and Ideas

  • Adversarial Robustness via Geometry: The authors show that the nonlinear JSR directly bounds the Lipschitz constant of a network with respect to the Thompson metric. This provides a more "geometrically natural" way to certify that a small change in input won't cause a massive change in output (adversarial robustness).
  • The "Sandwich" Bound: One of the most practical insights is Theorem 4.7: $\rho(F_\infty) \leq \rho(F) \leq \rho(F_0)$. This means you can bound the stability of a messy nonlinear system just by looking at its limits at zero and infinity.
  • Exploding Gradients: The framework offers a new perspective on the "exploding gradient" problem. By limiting the JSR of the layer-wise maps, researchers can theoretically guarantee that gradients won't blow up as the network gets deeper.

Key Results

  • Stability Equivalence: The paper proves that the system is asymptotically stable if and only if the nonlinear JSR is strictly less than 1 (Theorem 3.4).
  • Local vs. Global: They demonstrate that for this class of maps, the JSR controls not just the growth of the state, but also the divergence of trajectories starting from different points (Theorem 3.6).
  • Algorithm Convergence: The proposed algorithm successfully approximates the JSR for non-linear families, outperforming or extending standard linear methods when applied to matrices that preserve a cone.

Limitations and Future Work

  • The "Generalized" JSR Gap: In linear theory, the JSR is always equal to the "Generalized JSR" (the supremum of spectral radii of all finite compositions). In the nonlinear case, the authors show this equality only holds under specific conditions (e.g., the existence of an invariant subcone). Finding broader conditions for this equality remains an open question.
  • Computational Scaling: While the Polytopal Algorithm works for small dimensions, applying it to a production-scale neural network with millions of parameters is currently computationally prohibitive.
  • Beyond Cones: The current theory relies heavily on the structure of cones and order-preserving maps. Extending this to general nonlinear maps without these specific properties is a significant future challenge.

Summary updated

Abstract

We introduce a nonlinear extension of the joint spectral radius (JSR) for switched discrete-time dynamical systems governed by sub-homogeneous and order-preserving maps acting on cones. We show that this nonlinear JSR characterizes both the asymptotic stability of the system and the divergence or convergence rate of trajectories originating from different points within the cone. Our analysis establishes upper and lower bounds on the nonlinear JSR of a sub-homogeneous family via the JSRs of two associated homogeneous families obtained through asymptotic scaling. In the homogeneous case, we develop a dual formulation of the JSR and investigate the equality between the joint spectral radius and the generalized joint spectral radius, extending classical results from linear theory to the nonlinear setting. We also propose a polytopal-type algorithm to approximate the nonlinear JSR and provide conditions ensuring its finite-time convergence. The proposed framework is motivated by applications such as the analysis of deep neural networks, which can be modeled as switched systems with structured nonlinear layers. Our results offer new theoretical tools for studying the stability, robustness, and convergence behavior of such models.
BibTeX
@misc{deidda2025nonlinear,
  author = {Piero Deidda and Nicola Guglielmi and Francesco Tudisco},
  title = {{Nonlinear Joint Spectral Radius}},
  year = {2025},
  eprint = {2507.11314},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.11314},
  howpublished = {arXiv},
}
Figure