TL;DR
This paper tackles the challenge of catastrophic forgetting in continual learning by improving Experience Replay (ER). It introduces a novel metric, Batch Cosine Distance (BCD), to quantify inter-batch diversity and proposes a simple yet effective sampling strategy, Random Batch Sampling (RBS), that leverages this concept to significantly boost performance, especially in memory-constrained scenarios.
What Problem Does This Paper Address?
Continual Learning (CL) aims to train models that can learn new tasks sequentially without forgetting previously acquired knowledge. A major hurdle is "catastrophic forgetting," where learning a new task overwrites crucial information from older tasks. Experience Replay (ER) is a popular solution, where a memory buffer stores past samples to be replayed alongside new data. However, the effectiveness of ER hinges on the diversity of samples within these replay batches. Existing methods often focus on individual sample properties or overall memory diversity, but this paper highlights the importance of inter-batch diversity – how varied are the samples within a single replay batch – and proposes ways to measure and exploit it.
Methods
The paper introduces two key contributions:
- Batch Cosine Distance (BCD): This is a novel metric designed to measure the diversity of samples within a replay batch. It quantifies changes in hidden representations of samples before and after a model update. Essentially, it helps identify samples that are most affected by learning a new task and, by extension, measures the diversity of these affected regions in the model's feature space.
- Random Batch Sampling (RBS): This is a proof-of-concept sampling strategy that leverages the idea of inter-batch diversity. It involves randomly sampling a small number of batches from the memory buffer and selecting the batch that scores highest according to the BCD metric (or a similar diversity measure). This is a simple approach that aims to pick replay batches that are maximally diverse.
Datasets
The experiments were conducted on two standard datasets:
- MNIST: A dataset of handwritten digits.
- CIFAR-10: A dataset of small color images in 10 classes.
Both datasets were divided into five tasks, each with two classes, to simulate a continual learning scenario.
Key Insights and Ideas
- Inter-batch diversity matters: The paper argues that focusing on the diversity within a replay batch is crucial for effective experience replay, going beyond individual sample informativeness or overall memory representativeness.
- BCD as a novel diversity measure: The proposed Batch Cosine Distance metric offers a new way to quantify how diverse the impact of a replay batch is on the model's internal representations.
- Simplicity can be powerful: The RBS method, despite its straightforward approach of sampling diverse batches, demonstrates competitive performance against more complex methods.
- Memory size is critical: The benefits of diversity-aware sampling are particularly pronounced when the memory buffer is small, highlighting the importance of making the most of limited replay capacity.
Key Results
- Improved Accuracy: The proposed Random Batch Sampling (RBS) method consistently outperformed standard Experience Replay (ER) and the Maximally Interfered Retrieval (MIR) method on both CIFAR-10 and MNIST datasets across various memory buffer sizes.
- Significant Gains on CIFAR-10: For the CIFAR-10 dataset, RBS showed a more substantial improvement, especially with smaller memory sizes, demonstrating its effectiveness in memory-constrained settings.
- Competitive Performance on MNIST: On the MNIST dataset, RBS maintained competitive performance, though the differences between methods became less pronounced as memory size increased, a trend observed in other studies.
- Table 1 Comparison: The paper's earlier work (mentioned in the introduction) using Diverse Features Sampling (DFS) also showed improvements over random sampling and MIR, suggesting a general benefit from diversity-focused approaches.
Limitations and Future Work
- Computational Complexity: While DFS showed promise, its computational complexity was noted as a drawback. The RBS method, being simpler, likely mitigates some of this.
- Outlier Accumulation: DFS also had a tendency to accumulate outliers.
- Future Directions: The authors suggest several avenues for future work:
- Developing more sophisticated strategies to exclude less impactful samples.
- Integrating features from previous batches into selection criteria for a more context-aware approach.
- Exploring meta-learning methods to further enhance sample selection.
- Investigating strategies to ensure inter-batch diversity more broadly, potentially balancing the novel metric with other inherent properties of batches or sequences.
Summary updated