TL;DR
Weight decay ($L_2$ regularization) does more than just prevent overfitting—it actively biases neural networks toward learning low-rank weight matrices. This paper demonstrates that this low-rank bias explains the surprising success of model merging, showing that networks trained on different, nearly orthogonal tasks can be successfully combined by simply summing their weight matrices together.
What Problem Does This Paper Address?
Model merging—the practice of combining multiple trained neural networks by averaging or adding their weights—has become a popular and computationally cheap way to create multitask models. Despite its empirical success, we still lack a complete theoretical understanding of why it works. Why don't the parameters of two independently trained networks interfere with each other and destroy the model's performance when combined?
Previous deep learning theory often studied "implicit bias" under idealized settings without explicit regularization, or assumed the models achieved perfect zero training error (interpolation). This paper addresses this gap by analyzing the explicit role of $L_2$ regularization (weight decay) at stationary points. The authors connect weight decay to a low-rank structural bias and use it to provide a mathematical foundation for model merging.
Methods
The authors mathematically analyze depth-$K$ neural networks with $H$-positive homogeneous activations (such as ReLU or powers of ReLU) trained using logistic loss and $L_2$ regularization. They focus on stationary points—the limiting solutions where the gradient of the regularized loss function vanishes.
Key methodological contributions include:
- Characterizing Low-Rank Structure: The authors introduce a bound on the "pseudo-rank" (a weighted harmonic mean of the stable ranks across layers) and show it is explicitly controlled by the regularization parameter ($\lambda$) and the training loss.
- Proving Norm Preservation: They prove that at stationary points, the Frobenius norms of weight matrices are preserved across layers up to a scaling factor determined by the network's architecture.
- Model Merging Framework: They analyze a multitask setting where two networks are trained on distinct datasets with approximately orthogonal inputs. They provide bounds proving that the sum of these two networks' weights performs well on both tasks.
Datasets
The paper validates its theoretical claims using a mix of synthetic benchmarks, classic datasets, and large-scale pre-trained models:
- Synthetic Data: Inputs generated from isotropic Gaussians to simulate perfectly orthogonal tasks.
- Fashion MNIST: Adapted for binary classification and modified by appending/prepending zeros to ensure input orthogonality between different tasks.
- Pre-trained Transformers: To see if these theoretical properties hold in the wild, the authors evaluate several publicly available models: BERT (~110M parameters), RoBERTa (~125M), GPT2-Large (~774M), GPT2-XL (~1.5B), Phi-2 (~2.8B), and GPT-J (~6B).
Key Insights and Ideas
- Weight Decay is a Rank Compressor: The primary insight is that weight decay forces weight matrices to have a low stable rank, meaning their variance is concentrated across only a few dominant eigenvectors.
- Orthogonal Tasks Inhabit Non-Overlapping Subspaces: Because of this low-rank bias, when two networks are trained on nearly orthogonal tasks, their weight matrices learn to "reside" in entirely different, non-interfering subspaces of the parameter space.
- Summation vs. Averaging: While standard model merging often relies on parameter averaging for models trained on similar tasks, this paper reveals that for distinct, orthogonal tasks, parameter summation is the correct mathematical operation to preserve performance.
Key Results
- Low Rank in Large Language Models: Even though modern transformers violate several theoretical assumptions (e.g., they use LayerNorm and residual connections), empirical evaluations show they still exhibit remarkably low stable ranks across almost all layers.
- L2 Penalty Drives Down Rank: Experiments on synthetic data confirm that as the $L_2$ penalty increases, the average inverse stable rank increases (meaning the actual rank decreases), validating the core theoretical bound.
- Successful Merging: On both synthetic data and the modified Fashion MNIST dataset, summing the weights of networks trained on orthogonal tasks resulted in a merged model that achieved near-zero training loss and preserved generalization on both tasks simultaneously. Conversely, merging models trained on the same task via summation failed completely.
Limitations and Future Work
While the empirical results on LLMs are promising, the strict mathematical proofs rely on the assumption of positive homogeneity (e.g., pure ReLU networks without residual connections or normalization layers). Modern architectures break these assumptions. Additionally, the merging theory strictly requires the input spaces of the different tasks to be approximately orthogonal. Future work is needed to extend these proofs to non-orthogonal task distributions and more complex, heterogeneous network architectures.
Summary updated