Can neurogenesis act as a neural regularizer?

Lina Tran, Adam Santoro, Lulu Liu, Sheena A. Josselyn, Blake A. Richards, Paul W. FranklandbioRxiv 2022 · DOI

Abstract

New neurons are continuously generated in the subgranular zone of the dentate gyrus throughout adulthood. These new neurons gradually integrate into hippocampal circuits, forming new naïve synapses. Viewed from this perspective, these new neurons may represent a significant source of ‘wiring’ noise in hippocampal networks. In machine learning, such noise injection is commonly used as a regularization technique. Regularization techniques help prevent overfitting training data, and allow models to generalize learning to new, unseen data. Using a computational modeling approach, here we ask whether a neurogenesis-like process similarly acts as a regularizer, facilitating generalization in a category learning task. In a convolutional neural network (CNN) trained on the CIFAR-10 object recognition dataset, we modeled neurogenesis as a replacement/turnover mechanism, where weights for a randomly chosen small subset of neurons in a chosen hidden layer were re-initialized to new values as the model learned to categorize 10 different classes of objects. We found that neurogenesis enhanced generalization on unseen test data compared to networks with no neurogenesis. Moreover, neurogenic networks either outperformed or performed similarly to networks with conventional noise injection (i.e., dropout, weight decay, and neural noise). These results suggest that neurogenesis can enhance generalization in hippocampal learning through noise-injection, expanding on the roles that neurogenesis may have in cognition.

Introduction

The hippocampus is one of the few regions in the adult mammalian brain that maintains the capacity for neurogenesis—the lifelong birth and integration of new neurons. While traditional theories suggest these new cells increase memory capacity or facilitate pattern separation (the ability to distinguish between similar stimuli), the computational purpose of this constant circuit rewiring remains a subject of intense debate.

This research explores a novel hypothesis: that adult neurogenesis acts as a biological form of regularization. In artificial intelligence, regularization refers to techniques used to prevent "overfitting," a state where a model memorizes the specific noise and details of training data so thoroughly that it fails to generalize to new, unseen examples. By injecting "wiring noise" through the addition of new, naive neurons, the brain may intentionally disrupt its own established networks to ensure that memories remain flexible and generalized.

Methods and Modeling Approach

To test this hypothesis, the researchers employed a Convolutional Neural Network (CNN) trained on the CIFAR-10 dataset, a standard benchmark in computer vision consisting of 60,000 images across 10 categories (e.g., airplanes, cats, ships). The model architecture included three fully connected hidden layers, with the middle layer designated as the "Neurogenic Layer."

The Turnover Mechanism

Neurogenesis was simulated using a replacement/turnover mechanism:

  1. Turnover: Every 640 mini-batch updates, a small subset (3.2%, or 8 neurons) of the hidden layer was randomly selected for replacement.
  2. Re-initialization: The input and output weights of these selected neurons were reset to random values (using Kaiming uniform initialization). This effectively "retired" older, specialized neurons and replaced them with "new," naive ones.
  3. Benchmarking: The neurogenic network's performance was compared against a control group (no turnover) and three standard machine learning regularizers: Dropout (randomly silencing neurons), Weight Decay (penalizing large weight values), and Neural Noise (adding Gaussian noise to activations).

Key Insights

The Regularization Trade-off

The study found that neurogenic networks exhibited higher training loss but lower validation and test loss compared to controls. This is the classic signature of effective regularization: the model "sacrifices" perfect performance on the training set to ensure it learns the underlying patterns rather than specific image details.

Efficiency vs. Volume

While both neurogenesis and Dropout involve information loss, they operate at different scales. Dropout typically requires a high volume of noise (e.g., silencing 20% of neurons per pass) to be effective. In contrast, the neurogenic model achieved comparable or superior results with a much smaller footprint—only 3.2% turnover.

The Vulnerability Paradox

Interestingly, neurogenesis made the networks less resilient to ablation (the removal of neurons). While standard regularizers often encourage "distributed coding"—where many neurons share the informational load—neurogenesis seemed to push the network toward a more specialized structure where a few highly-tuned neurons became critical. This suggests that neurogenesis-driven regularization may result in a different internal representation than traditional AI methods.

Results

  • Generalization Boost: The neurogenic network achieved a test accuracy of 76.20%, significantly outperforming the control network’s 74.36%.
  • Competitive Performance: Neurogenesis performed as well as Dropout and significantly outperformed Weight Decay and Neural Noise within this specific architecture.
  • Biological Nuance: The researchers tested whether the "enhanced excitability" often seen in young biological neurons (a 30% boost in activity) would improve the model. Surprisingly, it did not, suggesting that the primary driver of the regularization effect is the turnover of weights itself.
  • Random vs. Targeted Replacement: Randomly replacing neurons was found to be just as effective as targeting "unimportant" neurons (those with low weight values). However, targeting "important" neurons for replacement severely degraded performance, highlighting the importance of preserving established high-value connections.

Discussion and Future Directions

These findings suggest that optimal levels of hippocampal neurogenesis may improve memory-guided decision-making by preventing the brain from overfitting to specific experiences. This promotes the formation of generalized memories applicable to a broader range of real-world circumstances.

While the CNN is an abstraction of the complex hippocampal circuit, the study provides a clear, testable prediction for behavioral neuroscience: suppressing neurogenesis in animal models should theoretically impair their ability to generalize in category learning tasks, such as recognizing a "car" regardless of its specific color or orientation. Future work may explore this turnover mechanism in more biologically plausible architectures, such as Recurrent Neural Networks (RNNs), to further bridge the gap between artificial and biological intelligence.

Summary updated

Abstract

Abstract New neurons are continuously generated in the subgranular zone of the dentate gyrus throughout adulthood. These new neurons gradually integrate into hippocampal circuits, forming new naïve synapses. Viewed from this perspective, these new neurons may represent a significant source of ‘wiring’ noise in hippocampal networks. In machine learning, such noise injection is commonly used as a regularization technique. Regularization techniques help prevent overfitting training data, and allow models to generalize learning to new, unseen data. Using a computational modeling approach, here we ask whether a neurogenesis-like process similarly acts as a regularizer, facilitating generalization in a category learning task. In a convolutional neural network (CNN) trained on the CIFAR-10 object recognition dataset, we modeled neurogenesis as a replacement/turnover mechanism, where weights for a randomly chosen small subset of neurons in a chosen hidden layer were re-initialized to new values as the model learned to categorize 10 different classes of objects. We found that neurogenesis enhanced generalization on unseen test data compared to networks with no neurogenesis. Moreover, neurogenic networks either outperformed or performed similarly to networks with conventional noise injection (i.e., dropout, weight decay, and neural noise). These results suggest that neurogenesis can enhance generalization in hippocampal learning through noise-injection, expanding on the roles that neurogenesis may have in cognition. Author Summary In deep neural networks, various forms of noise injection are used as regularization techniques to prevent overfitting and promote generalization on unseen test data. Here, we were interested in whether adult neurogenesis– the lifelong production of new neurons in the hippocampus– might similarly function as a regularizer in the brain. We explored this question computationally, assessing whether implementing a neurogenesis-like process in a hidden layer within a convolutional neural network trained in a category learning task would prevent overfitting and promote generalization. We found that neurogenesis regularization was as least as effective as, or more effective than, conventional regularizers (i.e., dropout, weight decay and neural noise) in improving model performance. These results suggest that optimal levels of hippocampal neurogenesis may improve memory-guided decision making by preventing overfitting, thereby promoting the formation of more generalized memories that can be applied in a broader range of circumstances. We outline how these predictions may be evaluated behaviorally in rodents with altered hippocampal neurogenesis.
BibTeX
@misc{tran2022can,
  author = {Lina Tran and Adam Santoro and Lulu Liu and Sheena A. Josselyn and Blake A. Richards and Paul W. Frankland},
  title = {{Can neurogenesis act as a neural regularizer?}},
  year = {2022},
  doi = {10.1101/2022.04.07.487582},
  howpublished = {bioRxiv},
}
Figure