The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions

Zheng Wang, Geyong Min, Wenjie Ruan

We’ve known for a while that Gradient Descent (GD) has a "secret sauce." Even when a neural network is massive enough to memorize every data point, GD steers it toward simple solutions that generalize well to new data. This is known as implicit bias. But while this bias is great for accuracy, what does it do for security?

A new paper, "The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions," reveals a startling trade-off: the very way GD trains layers to work together actually makes them more vulnerable to adversarial attacks.

TL;DR

As neural networks train, gradient descent forces adjacent layers to "align" their feature selection—a phenomenon the authors call co-correlation. While this collaboration helps the model learn, it creates a brittle chain where small input perturbations are easily amplified, systematically decreasing the model's adversarial robustness over time.

What Problem Does This Paper Address?

Deep learning models are notoriously vulnerable to adversarial examples—tiny, invisible tweaks to an image that can trick a classifier into thinking a school bus is an ostrich.

Researchers have long debated why this happens. Is it the data? The architecture? The training process? Most previous work on "implicit bias" focused on why models generalize so well. This paper shifts the lens to adversarial robustness. Specifically, it asks: How do the interactions between layers change during training, and how does that affect the model's ability to resist attacks?

Methods

To answer this, the authors introduced two key mathematical tools:

  1. Dirichlet Energy as a Metric: Instead of using the standard "Lipschitz constant" (which measures how much a function can change at most), they use Dirichlet energy to measure the average variability of the model. They theoretically prove that this energy is a superior proxy for adversarial risk.
  2. Co-correlation ($\varrho$): This is the paper's "aha" metric. It quantifies how much two successive layers "collaborate." If co-correlation is high (near 1), it means the second layer is perfectly tuned to amplify the specific features the first layer is looking for.

The researchers performed a dynamic analysis, mathematically proving how this co-correlation evolves at every step of gradient descent for both linear models and Multilayer Perceptrons (MLPs).

Key Insights and Ideas

  • The Collaboration Trap: We usually think of "collaboration" as a positive. However, in a neural network, high co-correlation means layers are becoming "tightly coupled." If the first layer is slightly fooled by a perturbation, the second layer—being highly aligned—will amplify that error rather than filtering it out.
  • Robustness Decays During Training: The authors show that co-correlation typically increases monotonically during training. This means that as a model gets better at its task (higher accuracy), it naturally becomes more "brittle" and easier to attack.
  • Width is a Shield: One of the most interesting findings is that wider networks are more robust because they resist this alignment. In a narrow network, layers must align perfectly to solve the problem. In a wide network, there is more "room" to find a solution without relying on extreme cross-layer collaboration, leading to a slower increase in co-correlation.
Co-correlation increases during training, but wider MLPs maintain lower co-correlation levels compared to narrow ones.

Key Results

  • The Robustness Gap: The researchers demonstrated a clear inverse relationship: as co-correlation goes up, robust accuracy (measured by the industry-standard "Auto-attack") goes down.
  • Width Matters: In experiments with MLPs, narrow models (e.g., width 16 or 32) showed a rapid spike in co-correlation. In contrast, very wide models (width 8,192) maintained much lower co-correlation and significantly higher robust accuracy.
  • Initialization Sensitivity: The "bias" toward collaboration is heavily influenced by how weights are initialized. By adjusting the variance of the initial weights (using a parameter $q$), the authors could actually "turn off" the increasing trend of co-correlation, though this often comes at the cost of training speed.
Robust accuracy across different network widths and weight initializations. Wider networks consistently show better robustness.

Limitations and Future Work

While the paper provides a rigorous foundation, there are a few hurdles for future research:

  • Architectural Complexity: Most of the formal proofs are for 2-layer MLPs. While the authors showed similar trends in ResNets, the math for complex structures like Transformers or Attention mechanisms remains an open challenge.
  • Input Constraints: The current theory assumes inputs are bounded by an $L_2$-norm, which might not capture the full complexity of real-world data distributions.
  • The Depth Factor: The study focuses on the interaction between adjacent layers. How this collaboration propagates through a 100-layer deep network is a promising direction for future exploration.

By identifying "co-correlation" as a hidden enemy of robustness, this work gives engineers a new metric to monitor during training—potentially leading to models that are not just accurate, but truly trustworthy.

Summary updated

Abstract

The implicit bias of gradient descent has long been considered the primary mechanism explaining the superior generalization of over-parameterized neural networks without overfitting, even when the training error is zero. However, the implicit bias toward adversarial robustness has rarely been considered in the research community, although it is crucial for the trustworthiness of machine learning models. To fill this gap, in this paper, we explore whether consecutive layers collaborate to strengthen adversarial robustness during gradient descent. By quantifying this collaboration between layers using our proposed concept, co-correlation, we demonstrate a monotonically increasing trend in co-correlation, which implies a decreasing trend in adversarial robustness during gradient descent. Additionally, we observe different behaviours between narrow and wide neural networks during gradient descent. We conducted extensive experiments that verified our proposed theorems.
BibTeX
@inproceedings{wang2024implicit,
  author = {Zheng Wang and Geyong Min and Wenjie Ruan},
  title = {{The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=jV6z08u7y0},
}
Figure