Abstract
High classification accuracy in Deep Neural Networks (DNNs) is often mistaken for system reliability. This paper demonstrates that learned weight matrices in recurrent networks are frequently "non-normal," meaning they are hyper-sensitive to tiny perturbations. While we can force these networks to be stable through spectral normalization, doing so often makes them more "brittle," highlighting a fundamental trade-off between stability, robustness, and performance.
Introduction: The Robustness Gap
In the development of Recurrent Neural Networks (RNNs), research has long focused on the "vanishing and exploding gradient" problem. Traditionally, the solution has been to manage the spectral radius—ensuring the largest eigenvalue of the weight matrix stays near 1 to prevent information from dissipating or uncontrolled growth over time.
However, this paper argues that mathematical stability (outputs not exploding) does not equate to structural robustness. A system can be stable yet remain hyper-sensitive to minor weight perturbations. This study investigates why highly accurate RNNs, LSTMs, and GRUs remain vulnerable to noise and adversarial attacks, specifically examining the "non-normality" of their weight matrices.
Methodology
The authors evaluate RNN, LSTM, and GRU architectures using tools from linear algebra and dynamical systems theory:
- Henrici’s Number: A metric for "non-normality." A normal matrix ($W^*W = WW^*$) has a Henrici number of zero; higher values indicate non-orthogonal eigenvectors, making the system sensitive to perturbations.
- $\epsilon$-pseudospectrum: A qualitative tool mapping how eigenvalues shift under small perturbations ($\epsilon$). This creates a "spectral portrait" of the network's robustness.
- Spectral Stabilization: Using the Power Method to constrain the spectral radius within the unit disk to re-evaluate robustness after ensuring stability.
- Schur Decomposition: Separating the weight matrix into its eigenvalues and its strictly upper triangular (non-normal) components.
Experimental Setup and Datasets
The study utilizes three benchmarks to test temporal learning:
- Adding Task: Requires the network to remember and sum specific numbers in long sequences.
- IMDB Sentiment Analysis: Classifying movie reviews as positive or negative.
- Sequential MNIST: Processing digit pixels one by one to test long-term memory.
Key Insights: The Nature of Non-Normality
- Accuracy $\neq$ Robustness: Networks can achieve 99% accuracy while possessing highly non-normal and unstable weight matrices. High performance is not a proxy for safety.
- Non-normality is the Norm: Training via backpropagation naturally results in non-normal matrices where eigenvectors may not be orthogonal, leading to transient growth and instability.
- The Edge of Criticality: To "remember" long sequences, Jacobian eigenvalues must approach the unit circle. This "edge of chaos" allows for high information processing but results in extreme fragility.
Key Results: The Stability-Robustness Paradox
The paper's most counter-intuitive finding is the "brittle" nature of normalization. When spectral normalization was used to force stability, the $\epsilon$-pseudospectrum showed that the networks actually became more brittle. Tiny perturbations caused significantly larger shifts in the spectral portrait than in un-normalized networks.
Quantitatively, significant Henrici numbers were found across all tasks. In the IMDB task, LSTMs and GRUs showed high non-normality across internal gates. While RNNs and GRUs performed well on Sequential MNIST (19-26% accuracy), LSTMs often failed to converge, highlighting the difficulty of training these complex dynamical systems.
Conclusion and Future Directions
Constraining the eigen-spectrum is insufficient for safety-critical AI. The authors propose two paths forward:
- Simultaneous Optimization: Developing algorithms that optimize for performance, stability, and robustness concurrently using pseudospectral tools.
- The Variational Approach: Leveraging the Free Energy Principle and Variational Bayes to find a balance between complexity and accuracy, potentially leading to solutions that are inherently more robust to noise.
Until the underlying linear algebra of weight matrices is addressed, deep learning systems will remain fundamentally brittle despite their surface-level accuracy.
Summary updated