Stochastic Gradient Descent-induced drift of representation in a two-layer neural network

Farhad Pashakhanloo, Alexei A. Koulakov

Introduction

In both biological and artificial neural networks, a fundamental question persists: once a system has learned a task, does its internal representation of information remain stable? Evidence from neuroscience suggests otherwise. In the mammalian olfactory system, for instance, neural activity patterns representing specific scents have been observed to change over time—a phenomenon known as representational drift—even while the animal’s behavioral response to those scents remains perfectly consistent.

In their paper, Farhad Pashakhanloo and Alexei Koulakov provide a rigorous mathematical framework to explain this drift. By analyzing a two-layer linear neural network trained via Stochastic Gradient Descent (SGD), the authors demonstrate that representational drift is not merely a biological quirk but an inevitable consequence of learning dynamics in systems with redundant parameters. Their work bridges the gap between the stochastic nature of machine learning and experimental observations in sensory neuroscience.

The Problem of Redundancy and Stability

In an overparameterized neural network, there is rarely a single "correct" set of weights. Instead, there exists a high-dimensional manifold of solutions—a surface in the parameter space where every point corresponds to a state of minimum loss. Once a network reaches this manifold, it has technically "solved" the task.

However, the presence of noise during the learning process—specifically the sampling noise inherent in SGD—prevents the network from sitting still. Previous theories suggested that synaptic noise or random fluctuations might drive drift, but these models often failed to explain why certain stimuli drift faster than others. Specifically, biological data shows that familiar or frequently encountered stimuli exhibit slower representational drift than novel ones. Pashakhanloo and Koulakov sought to determine if the mechanics of SGD alone could account for this discrepancy.

Methodological Framework

The researchers utilized a two-layer linear feedforward network configured as an autoencoder, tasked with learning an identity mapping (outputting the exact input it receives). To analyze the movement of the network's weights, they decomposed the learning dynamics into two distinct components:

  1. Normal Space (Fluctuations): These are movements perpendicular to the solution manifold. When SGD noise pushes the weights into this space, the gradient of the loss function acts as a "restoring force," pulling the network back toward the manifold.
  2. Tangent Space (Diffusion): These are movements along the manifold itself. Because every point on the manifold represents an equivalent solution, there is no restoring force in this direction. Consequently, the network undergoes a random-walk-like diffusion, which manifests as representational drift.

By treating the SGD updates as a continuous stochastic process through Stochastic Differential Equations (SDEs), the authors were able to analytically calculate the diffusion coefficients that govern how quickly representations move along this manifold.

Key Insights: Drift as a Second-Order Phenomenon

A central finding of the study is that representational drift is a "second-order" phenomenon. On the solution manifold, the gradient of the loss function is zero. Therefore, the "push" that causes the network to slide along the manifold only occurs when the network is first knocked off the manifold by SGD noise.

This leads to a crucial realization regarding stimulus frequency. The authors proved that frequent stimuli cause smaller fluctuations in the "normal" space. Because these fluctuations are the primary drivers of diffusion in the "tangent" space, stimuli that appear more often naturally drift at a slower rate. This provides a purely mathematical explanation for the biological observation that familiarity breeds representational stability.

Furthermore, the study addresses how a network can maintain a stable output despite its internal layers being in constant flux. In the two-layer model, the drift in the first layer of weights is compensated for by a corresponding drift in the second layer. These changes effectively cancel each other out relative to the final output, ensuring that the "readout" remains stable even as the internal "code" evolves.

Results and Implications

The theoretical predictions were validated through numerical simulations. The authors derived closed-form equations for the drift rate and found an almost perfect match with the results of actual SGD training. Their analysis confirmed that the diffusion coefficient for a frequent stimulus ($D_a$) is strictly lower than that of background or infrequent stimuli ($D_b$).

This work has significant implications for both AI and neuroscience:

  • For AI: It suggests that representational drift is an inherent feature of any redundant system trained with stochastic methods. Understanding this drift could lead to better regularization techniques or more robust continual learning algorithms.
  • For Neuroscience: It provides a parsimonious explanation for drift that does not require specialized biological mechanisms, suggesting that the "noise" of learning itself is sufficient to explain the brain's shifting neural codes.

Limitations and Future Directions

While the two-layer linear model provides a clear analytical window into drift dynamics, it is a simplification of the highly nonlinear processes found in deep learning and biological brains. Future research is needed to explore how nonlinear activation functions and more complex tasks, such as multi-class classification, influence the geometry of the solution manifold. Additionally, investigating how non-stationary data distributions—where the environment itself changes—affects these dynamics will be a vital next step in understanding the long-term evolution of neural representations.

Summary updated

BibTeX
@inproceedings{pashakhanloo2023stochastic,
  author = {Farhad Pashakhanloo and Alexei A. Koulakov},
  title = {{Stochastic Gradient Descent-induced drift of representation in a two-layer neural network}},
  booktitle = {International Conference on Machine Learning},
  year = {2023},
  url = {https://openreview.net/forum?id=P2Y3ykdwWN},
}
Figure