Unsupervised Replay Strategies for Continual Learning with Limited Data

Anthony Bazhenov, Pahan Dewasurendra, Giri P. Krishnan, Jean Erik Delanois

TL;DR

This paper introduces an unsupervised "sleep replay" mechanism (SRC) for artificial neural networks (ANNs) that significantly improves performance on tasks with limited or imbalanced data, and crucially, helps prevent catastrophic forgetting in continual learning scenarios. By mimicking biological sleep's memory consolidation, SRC enhances learning efficiency and rescues previously learned information without requiring additional labeled data.

What Problem Does This Paper Address?

Artificial neural networks (ANNs) excel in many tasks but struggle with two critical limitations: they require vast amounts of data to learn effectively, and they suffer from "catastrophic forgetting" when trained sequentially on new tasks, losing previously acquired knowledge. This is in stark contrast to human learning, which is efficient, continuous, and can learn from very few examples. While existing methods address either limited data or catastrophic forgetting, they often don't work well together. This paper aims to bridge this gap by exploring an unsupervised approach inspired by biological sleep's memory replay to improve both limited-data learning and continual learning.

Methods

The core of this research is the Sleep Replay Consolidation (SRC) algorithm. When an ANN is trained on limited data, it's first converted into a spiking neural network (SNN). This SNN is then subjected to an unsupervised "sleep" phase. During this phase, the network is stimulated with Poisson-distributed spike trains that mimic input data. Crucially, synaptic weights are adjusted using a local, unsupervised Hebbian-like learning rule. This rule strengthens connections where pre- and post-synaptic neurons are co-activated and weakens them otherwise. After the sleep phase, the SNN is converted back to an ANN. This process is designed to reactivate and consolidate relevant memories without explicit supervision. The authors also explored fine-tuning the model after the sleep phase.

Datasets

The study utilized two widely-used image datasets:

  • MNIST: A dataset of handwritten digits (0-9).
  • Fashion MNIST (FMNIST): A dataset of Zalando's article images, representing 10 different clothing categories.

For their experiments, the researchers created specific data scenarios:

  • Limited Data: Subsets of 0.1% to 10% of the full datasets were used.
  • Imbalanced Data: Within limited subsets, the number of images for one class was intentionally reduced, creating an imbalance.
  • Continual Learning: Tasks were designed to distinguish between classes 0-4 (Task 1) and then classes 5-9 (Task 2), with varying amounts of data for each task.

Key Insights and Ideas

  • Sleep as a Universal Enhancer: The "sleep replay" mechanism (SRC) proved effective across different challenges: limited data, imbalanced data, and sequential learning. This suggests a fundamental role for this unsupervised replay in improving ANN learning.
  • Beyond Simple Rehearsal: Unlike traditional rehearsal methods that store and replay explicit past data, SRC uses an unsupervised, biologically inspired process to reactivate and consolidate learned patterns within the network's existing structure.
  • Mitigating Catastrophic Forgetting: SRC not only rescued performance on older tasks after training on new ones but sometimes even enhanced performance on those older tasks, a significant improvement over standard sequential training.
  • Class-Balanced Learning: For imbalanced datasets, SRC helped improve the accuracy of under-represented classes, leading to a more balanced and fair model.
  • Network Dynamics: Analysis revealed that SRC primarily modifies hidden layers, leading to sparser and more contrasting representations, which likely contributes to improved generalization and reduced interference.

Key Results

  • Limited Data Performance: SRC significantly improved accuracy on MNIST and FMNIST datasets when using as little as 0.2% to 10% of the training data (Figure 1). For example, on a 3% MNIST subset, overall accuracy rose from 0.32 to 0.60 after SRC, with notable improvements in previously underperforming classes.
  • Imbalanced Data Performance: Figure 4 shows that SRC generally led to positive improvements in class-wise accuracy for under-represented classes, even when data reduction was substantial.
  • Continual Learning: In sequential learning tasks, SRC rescued performance on Task 1 after training on Task 2, especially when data was limited. Figure 6 shows significant gains in overall accuracy (15-30% for MNIST, 5-15% for FMNIST) when using equal amounts of data for both tasks.
  • Fine-tuning Synergy: While SRC alone improved performance, a short fine-tuning phase after SRC further boosted accuracy, particularly in low-data regimes.

Limitations and Future Work

  • Performance Degradation at High Data: The study observed a slight decrease in performance (10-15%) when more than approximately 10% of the data was used for training. The authors suggest this could be mitigated by a short fine-tuning phase, which was explored.
  • Hyperparameter Sensitivity: While SRC is presented as a general method, the specific hyperparameters used were determined via a genetic algorithm for optimal performance on the tested datasets. Further research could explore more robust or adaptive hyperparameter tuning.
  • Generalizability to Other Architectures/Tasks: The current study focused on fully-connected feed-forward networks and image datasets. Future work could investigate SRC's effectiveness on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and other types of data or tasks.
  • Understanding the "Why": While the paper provides insights into network dynamics (e.g., sparser representations), a deeper theoretical understanding of how SRC precisely achieves its benefits across various scenarios would be valuable.

Summary updated

Abstract

Artificial neural networks (ANNs) show limited performance with scarce or imbalanced training data and face challenges with continuous learning, such as forgetting previously learned data after new tasks training. In contrast, the human brain can learn continuously and from just a few examples. This research explores the impact of 'sleep', an unsupervised phase incorporating stochastic activation with local Hebbian learning rules, on ANNs trained incrementally with limited and imbalanced datasets, specifically MNIST and Fashion MNIST. We discovered that introducing a sleep phase significantly enhanced accuracy in models trained with limited data. When a few tasks were trained sequentially, sleep replay not only rescued previously learned information that had been catastrophically forgetting following new task training but often enhanced performance in prior tasks, especially those trained with limited data. This study highlights the multifaceted role of sleep replay in augmenting learning efficiency and facilitating continual learning in ANNs.
BibTeX
@inproceedings{bazhenov2024unsupervised,
  author = {Anthony Bazhenov and Pahan Dewasurendra and Giri P. Krishnan and Jean Erik Delanois},
  title = {{Unsupervised Replay Strategies for Continual Learning with Limited Data}},
  booktitle = {IJCNN},
  year = {2024},
  doi = {10.1109/ijcnn60899.2024.10650116},
}
Figure