REPAIR: REnormalizing Permuted Activations for Interpolation Repair

Keller Jordan, Hanie Sedghi, Olga Saukh, Rahim Entezari, Behnam Neyshabur

TL;DR

Researchers have discovered that the primary reason deep neural networks perform poorly when their weights are linearly interpolated is a phenomenon called variance collapse. By introducing REPAIR, a simple method to rescale activation statistics, they can reduce the performance "barrier" between merged models by up to 90%, enabling more effective model ensembling and merging.

What Problem Does This Paper Address?

Can you take two neural networks trained independently on the same task and simply average their weights to get a single, high-performing model? Usually, the answer is a resounding "no." Even if you account for the fact that neurons can be permuted (the "permutation invariance" problem), the resulting interpolated model typically suffers from a massive drop in accuracy, known as a loss barrier.

Previous work conjectured that if you align the neurons correctly, this barrier should disappear. However, in practice, while alignment helps shallow networks, deep networks like ResNets still fail miserably when averaged. This paper identifies why this happens and provides a fix.

Methods

The authors identify the culprit as variance collapse. When you average the weights of two networks, the activations in the resulting model have significantly lower variance than the original models. As you go deeper into the network, this effect compounds until the activations become nearly constant, leaving the network unable to distinguish between different inputs.

Variance collapse across layers in different architectures. Before REPAIR (open markers), variance drops sharply in deeper layers.

To fix this, they propose REPAIR (REnormalizing Permuted Activations for Interpolation Repair). The process works in two main steps:

  1. Neuron Alignment: Use standard methods (like activation matching) to permute the neurons of one network so they "line up" with the other.
  2. Statistical Correction (The "Repair"): They introduce a forward-pass method to track the mean and variance of activations in the endpoint networks. They then apply a per-channel rescale-and-shift (essentially a specialized Batch Normalization) to the interpolated network to ensure its activation statistics match the expected average of the two original models.

Datasets

The authors evaluated REPAIR across a wide range of standard benchmarks:

  • MNIST & FashionMNIST: For initial MLP and ResNet20 experiments.
  • CIFAR-10 & CIFAR-100: Used for VGG and ResNet architectures, including experiments on merging models trained on disjoint data splits.
  • SVHN: For testing across different data distributions.
  • ImageNet: The ultimate test for scaling to large-scale, complex visual recognition.

Key Insights and Ideas

  • The "Aha" Moment - Variance Collapse: The core insight is that interpolating activations $X_\alpha = (1-\alpha)X_1 + \alpha X_2$ naturally reduces variance unless the two variables are perfectly correlated. Since aligned neurons in different models usually have a correlation of only ~0.4, the variance of the averaged model drops by about 30% per layer. In a deep network, this compounds until the signal vanishes.
  • LayerNorm vs. BatchNorm: The paper elegantly explains a contradiction in previous literature. Some researchers found low barriers in ResNets, while others didn't. The authors realized the "successful" cases had replaced standard Batch Normalization with Layer Normalization. Because LayerNorm renormalizes at test time, it accidentally mitigates variance collapse, whereas BatchNorm (which uses fixed running statistics) does not.
  • Constructive Merging: REPAIR allows for "constructive merging," where two models trained on different subsets of data (e.g., one good at classes 1-50, another at 51-100) can be averaged into a single model that outperforms both on the full dataset.

Key Results

REPAIR consistently leads to massive reductions in the interpolation barrier across architectures and tasks.

  • ResNet50 on ImageNet: Without REPAIR, an averaged model has <1% accuracy. With REPAIR, the barrier is reduced by 74%, boosting midpoint accuracy to 56.5%.
  • ResNet18 on CIFAR-10: Achieved a 90% relative barrier reduction, bringing the interpolation barrier down to just 1.5%.
  • Scaling with Width: The authors found that wider networks generally have lower barriers after REPAIR, suggesting that overparameterization makes the loss landscape more amenable to linear interpolation.
REPAIR significantly boosts the accuracy of interpolated ResNet50s on ImageNet.

Limitations and Future Work

While REPAIR is highly effective, it isn't a "magic bullet" for zero-barrier connectivity in all cases. On ImageNet, even with REPAIR, a significant barrier remains (20.1% test error increase for ResNet50).

Furthermore, REPAIR requires a forward pass through a small amount of training data to calculate the necessary statistics, making it a post-hoc correction rather than a change to the training objective itself. Future work could explore how to train models specifically to be "mergable" without requiring this secondary step, or how to apply these insights to more complex architectures like Transformers.

Summary updated

Abstract

In this paper we look into the conjecture of Entezari et al. (2021) which states that if the permutation invariance of neural networks is taken into account, then there is likely no loss barrier to the linear interpolation between SGD solutions. First, we observe that neuron alignment methods alone are insufficient to establish low-barrier linear connectivity between SGD solutions due to a phenomenon we call variance collapse: interpolated deep networks suffer a collapse in the variance of their activations, causing poor performance. Next, we propose REPAIR (REnormalizing Permuted Activations for Interpolation Repair) which mitigates variance collapse by rescaling the preactivations of such interpolated networks. We explore the interaction between our method and the choice of normalization layer, network width, and depth, and demonstrate that using REPAIR on top of neuron alignment methods leads to 60%-100% relative barrier reduction across a wide variety of architecture families and tasks. In particular, we report a 74% barrier reduction for ResNet50 on ImageNet and 90% barrier reduction for ResNet18 on CIFAR10.
BibTeX
@inproceedings{jordan2023repair,
  author = {Keller Jordan and Hanie Sedghi and Olga Saukh and Rahim Entezari and Behnam Neyshabur},
  title = {{REPAIR: REnormalizing Permuted Activations for Interpolation Repair}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=gU5sJ6ZggcX},
}
Figure