Neural networks are often viewed as high-dimensional "black boxes" where millions of parameters shift in unison. However, a growing body of research suggests that the complexity of these models can be distilled by decomposing their weights into two competing components: a signal component that captures generalizable patterns and a noise component that memorizes sample-specific artifacts.
This framework, known as signal-noise weight decomposition, provides a mathematical lens to track how models learn, why they generalize, and why certain optimizers outperform others.
The Spiked Weight Matrix
At the heart of this idea is the observation that a trained network's weights are not purely random. Instead, they often resemble a "spiked" random matrix. As Cui et al. and Yoo et al. describe, the weight matrix can be modeled as a combination of the initial random weights (noise) and a low-rank "spike" that emerges during training. This spike represents the specific directions in the weight space that have aligned with the target function's features.
By isolating this spike, researchers can distinguish between the underlying random feature map and the representational changes driven by gradient descent. This separation is crucial for understanding how networks "break out" of the lazy training regime to capture complex non-linear features Cui et al..
Tracking Learning via Data Patches
To make this decomposition mathematically precise, researchers often employ a structural data model. In this model, inputs are decomposed into signal patches—informative features correlated with the label—and noise patches—high-dimensional random fluctuations.
Zou et al. and Tang et al. use this patch-based framework to track the alignment between model weights and different parts of the input. By calculating the inner product of the weights with these patches throughout training, they can monitor whether an optimizer like Adam is prioritizing the learning of shared structural signals or the memorization of random noise. This granular view reveals why Adam sometimes generalizes poorly compared to SGD: its update rules can inadvertently accelerate the growth of the noise-aligned weight components Tang et al..
The Two-Stage Dynamics of Training
Signal-noise decomposition also reveals a distinct temporal structure in training. Han et al. demonstrate that neural networks typically undergo a two-stage process:
- Signal Learning: In the early stages, the model updates are dominated by coefficients that track consistent, label-dependent features.
- Noise Memorization: As training continues, the model pivots to memorize sample-specific noise to drive the training error to zero.
This mathematical "why" explains the success of practical heuristics like early stopping. Furthermore, Huang et al. show that techniques like injecting label noise can act as regularizers by suppressing the growth of the noise component, allowing the signal component to grow exponentially even in low signal-to-noise ratio (SNR) settings.
Explaining Benign Overfitting
One of the most counterintuitive phenomena in deep learning is benign overfitting: the ability of a model to perfectly fit noisy training data while still generalizing well to new data.
Kou et al. and George et al. use signal-noise decomposition to prove that generalization depends on the relative strength of these two components. If the signal-to-noise ratio of the learned weights remains high, the model's decision boundary will be dominated by the underlying signal rather than the memorized noise. George et al. show that this can occur even when the input dimension is only proportional to the sample size, provided the optimization process maximizes the margin of the signal component.
Applications Across Architectures
The utility of this decomposition extends beyond standard feed-forward networks:
- Graph Neural Networks (GNNs): Huang et al. quantify the advantage of GNNs over MLPs by showing that graph convolutions significantly prioritize signal learning over noise memorization.
- Transformers: Li et al. use feature learning theory to show that the attention mechanism in transformers can be prone to memorizing noise when trained with certain optimizers like SignGD.
- Adversarial Robustness: Haldar et al. decompose weights into "authentic" components (aligned with the data manifold) and "volatile" components (off-manifold). They prove that adversarial vulnerability is driven by the growth of these volatile components, which occupy the gap between the data's intrinsic and ambient dimensions.
By treating the weight space as a battlefield between signal and noise, this research direction provides a unified theoretical framework for understanding the hidden mechanics of modern machine learning.
Key Foundational Work
Removing the Feature Correlation Effect of Multiplicative Noise (2018) — contributed 1 concept elements to this direction.
Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization (2020) — contributed 1 concept elements to this direction.
iNNvestigate neural networks! (2018) — contributed 1 concept elements to this direction.