Reviewer #2 (Public Review): Representational drift as a result of implicit regularization

Aviv Ratzon, Dori Derdikman, Omri BarakOpenalex Unknown 2024 · DOI · OpenAlex

Abstract

Recent studies show that, even in constant environments, the tuning of single neurons changes over time in a variety of brain regions. This representational drift has been suggested to be a consequence of continuous learning under noise, but its properties are still not fully understood. To investigate the underlying mechanism, we trained an artificial network on a simplified navigational task. The network quickly reached a state of high performance, and many units exhibited spatial tuning. We then continued training the network and noticed that the activity became sparser with time. Initial learning was orders of magnitude faster than ensuing sparsification. This sparsification is consistent with recent results in machine learning, in which networks slowly move within their solution space until they reach a flat area of the loss function. We analyzed four datasets from different labs, all demonstrating that CA1 neurons become sparser and more spatially informative with exposure to the same environment. We conclude that learning is divided into three overlapping phases: (i) Fast familiarity with the environment; (ii) slow implicit regularization; (iii) a steady state of null drift. The variability in drift dynamics opens the possibility of inferring learning algorithms from observations of drift statistics.

Introduction: The Mystery of Representational Drift

In neuroscience, "representational drift" describes a puzzling phenomenon: neural activity patterns change over days or weeks even when an animal performs a well-learned task in a stable environment. If behavior is already optimized, why does the brain continue to shift its internal representations?

Traditional theories often characterize drift as an undirected "random walk" through a space of functionally equivalent solutions. However, this view fails to explain why certain statistical features of neural activity, such as sparsity, evolve systematically over time. This research bridges neuroscience and machine learning (ML) theory to propose that drift is a directed process of implicit regularization, pushing neural codes toward more robust and efficient configurations.

Modeling Drift as Implicit Regularization

To test this hypothesis, the authors trained artificial neural networks (ANNs) on a predictive coding task where an agent navigates a corridor. By introducing noisy learning rules—simulating biological constraints—they observed how representations evolved after initial task mastery.

Conceptual model of representational drift. (A) The loss landscape illustrates how noise drives the system toward flatter regions of the solution space. (B) Temporal evolution of a property (such as sparsity) under undirected "null" drift (left) versus directed implicit regularization (right).

The study found that learning does not terminate once the error is minimized. Instead, persistent noise in the learning process drives the network along the "solution manifold" toward regions where the loss landscape is flatter. In the context of hippocampal-like architectures, this movement manifests as spontaneous sparsification: unnecessary units are deactivated, resulting in a more efficient code.

Key Insights: The Three Phases of Learning

The authors propose that learning occurs in three distinct, overlapping stages:

  1. Fast Familiarity: The network rapidly achieves high performance on the task.
  2. Slow Implicit Regularization: A prolonged period of directed drift where representations become sparser and more spatially informative.
  3. Steady State Null Drift: A final phase where the network has reached maximum "flatness" and undergoes undirected random fluctuations.

Crucially, as the number of active neurons decreases during the regularization phase, the remaining active neurons become more spatially informative, enhancing the quality of the representation despite the reduced cell count.

Experimental Validation and Biological Consistency

The model's predictions were validated through a meta-analysis of four independent hippocampal (CA1) datasets from different laboratories (Khatib et al., 2023; Jercog et al., 2019; Karlsson et al., 2015; Sheintuch et al., 2023). Across all datasets, rodents becoming familiar with an environment showed a consistent decrease in the fraction of active place cells alongside an increase in Mean Spatial Information (SI) per cell. Mathematical analysis confirmed that this sparsification is directly linked to a reduction in the eigenvalues of the Hessian matrix, a hallmark of finding flatter, more robust solutions in ML.

Conclusion and Future Directions

This work reinterprets representational drift not as noise, but as a signature of the brain's ongoing optimization process. While the specific signature observed here was sparsification, different network architectures or noise types might produce different regularizing effects.

Future research could leverage these drift statistics to reverse-engineer the specific learning algorithms used by the brain. Furthermore, exploring how this drive toward flatness interacts with "remapping"—the creation of new representations for new environments—could provide deeper insights into the flexibility and stability of biological memory systems.

Summary updated

Abstract

Recent studies show that, even in constant environments, the tuning of single neurons changes over time in a variety of brain regions. This representational drift has been suggested to be a consequence of continuous learning under noise, but its properties are still not fully understood. To investigate the underlying mechanism, we trained an artificial network on a simplified navigational task. The network quickly reached a state of high performance, and many units exhibited spatial tuning. We then continued training the network and noticed that the activity became sparser with time. Initial learning was orders of magnitude faster than ensuing sparsification. This sparsification is consistent with recent results in machine learning, in which networks slowly move within their solution space until they reach a flat area of the loss function. We analyzed four datasets from different labs, all demonstrating that CA1 neurons become sparser and more spatially informative with exposure to the same environment. We conclude that learning is divided into three overlapping phases: (i) Fast familiarity with the environment; (ii) slow implicit regularization; (iii) a steady state of null drift. The variability in drift dynamics opens the possibility of inferring learning algorithms from observations of drift statistics.
BibTeX
@inproceedings{ratzon2024reviewer,
  author = {Aviv Ratzon and Dori Derdikman and Omri Barak},
  title = {{Reviewer \#2 (Public Review): Representational drift as a result of implicit regularization}},
  booktitle = {Openalex Unknown},
  year = {2024},
  doi = {10.7554/elife.90069.2.sa1},
}
Figure