Truth or backpropaganda? An empirical investigation of deep learning theory

Micah Goldblum, Jonas Geiping, Avi Schwarzschild, Michael Moeller, Tom Goldstein

In a "myth-busters" style investigation, Truth or backpropaganda? tests whether common mathematical assumptions in deep learning theory actually hold up in practical settings. The authors demonstrate that suboptimal local minima are widespread, small-norm parameters aren't always better for generalization, and modern architectures like ResNets fundamentally defy current "wide-network" theories.

The Wedge Between Theory and Practice

A growing divide exists between deep learning theory and practice. To make mathematical proofs tractable, theorists often study "toy" models: extremely wide networks, linear activations, or networks without skip connections. These simplified studies lead to widely held beliefs—for instance, that all local minima are "good enough" or that low-rank matrices inherently improve robustness.

The authors argue that these theoretical conclusions often fail when applied to the realistic architectures (like ResNets) and datasets (like CIFAR-10) that engineers actually use. They aim to identify where theory accurately describes reality and where it falls into the realm of "backpropaganda."

Experimental Approach

The researchers use a combination of mathematical proofs and empirical "stress tests" across four key areas, primarily utilizing standard computer vision benchmarks like CIFAR-10 and CIFAR-100:

  1. Local Minima Construction: They prove that Multi-Layer Perceptrons (MLPs) inherit the suboptimal local minima of linear classifiers. They then use specific initializations (like very high bias variance) to "trap" realistic networks in these bad minima.
  2. Norm-Bias Regularization: To test if small parameter norms are truly optimal, they propose a new regularizer called norm-bias. Instead of pushing weights toward zero (like standard weight decay), it biases them toward a specific non-zero norm ($\mu$).
  3. NTK Correlation Analysis: They track the Neural Tangent Kernel (NTK)—a mathematical tool used to describe how networks learn in the "infinite width" limit. They measure the correlation between the kernel at initialization and after training across different widths and architectures.
  4. Rank Manipulation (RankMin/RankMax): They employ specialized training protocols to force network layers to be either low-rank or high-rank to see how these properties correlate with accuracy and adversarial robustness.

Key Insights and Findings

Bad Minima are Real (but Sharp)

While many theorists argue that "all local minima are global," the authors prove and find "bad" minima with high training loss. By initializing with high-variance bias, they found local minima for MLPs on CIFAR-10 with a training loss of 2.31, compared to the near-zero loss (0.006) found with default initialization. Interestingly, these suboptimal minima are significantly "sharper" (higher Hessian eigenvalues) than the good ones, suggesting that standard optimizers avoid them not because they don't exist, but because they reside in narrow, hard-to-hit basins.

The "Small Norm" Myth

Classical theory suggests that smaller weights lead to better generalization (Tikhonov regularization). However, the authors found that biasing a ResNet-18 toward a larger norm actually improved performance. On CIFAR-10, a ResNet-18 with standard weight decay achieved 94.76% accuracy, while the same model using "norm-bias" (biasing toward a larger-than-zero norm) achieved a superior 94.99%. This suggests that the benefit of weight decay might be about regulating the "effective" learning rate rather than just keeping the norm small.

ResNets Defy the Wide-Network Limit

In simple MLPs, as you make the network wider, the NTK stays more constant during training, which matches theoretical expectations. In ResNets, the opposite happens: the kernel changes more as the network gets wider.

NTK metrics across architectures as width increases. Notice how WideResNet (green) diverges from theoretical expectations, showing decreasing correlation and increasing change in NTK.

For ConvNets, the correlation between initial and final kernels approaches 1.0 as width increases. For WideResNets, this correlation actually drops as width increases, showing a fundamental divergence from NTK theory. The authors trace this defiance to the interaction between skip connections and Batch Normalization.

Rank Doesn't Guarantee Robustness

Contrary to recent claims that low-rank layers make networks more robust to attacks, the authors found that "RankMax" (purposefully high-rank) models often outperformed "RankMin" models in both clean accuracy and adversarial robustness.

Conclusion and Future Work

The authors acknowledge that while they found counter-examples to many theories, they haven't yet proposed a "unified theory" to replace them. Furthermore, the study is primarily focused on image classification; it remains to be seen if these theoretical failures look the same in Transformers or Reinforcement Learning agents.

Ultimately, the paper calls for theorists to move past toy models and specifically investigate why the combination of skip connections and normalization—the "bread and butter" of modern AI—behaves so differently from the models currently being proven in academic literature.

Summary updated

Abstract

We empirically evaluate common assumptions about neural networks that are widely held by practitioners and theorists alike. In this work, we: (1) prove the widespread existence of suboptimal local minima in the loss landscape of neural networks, and we use our theory to find examples; (2) show that small-norm parameters are not optimal for generalization; (3) demonstrate that ResNets do not conform to wide-network theories, such as the neural tangent kernel, and that the interaction between skip connections and batch normalization plays a role; (4) find that rank does not correlate with generalization or robustness in a practical setting.
BibTeX
@inproceedings{goldblum2020truth,
  author = {Micah Goldblum and Jonas Geiping and Avi Schwarzschild and Michael Moeller and Tom Goldstein},
  title = {{Truth or backpropaganda? An empirical investigation of deep learning theory}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=LidbmSbyrxs},
}
Figure