TL;DR
This paper tackles the "catastrophic forgetting" problem in deep learning, where models forget previously learned tasks when trained on new ones. They propose a novel "Kalman Optimiser" that intelligently divides a neural network into long-term and short-term memory units, allowing it to learn new tasks without sacrificing performance on old ones.
What Problem Does This Paper Address?
Deep learning models excel at many tasks, but they struggle with sequential learning. When a model is trained on a new dataset or task, its parameters are updated, often leading to a significant degradation in performance on previously learned tasks. This phenomenon, known as "catastrophic forgetting," makes it difficult to deploy models in dynamic, real-world environments where new information is constantly encountered. Traditional approaches often involve retraining the entire model, using separate models for each task, or storing all past data, which are computationally expensive or impractical.
Methods
The core of this paper is the proposed Kalman Optimiser. This approach treats neural network weights not as fixed values but as indeterminate quantities with associated uncertainties. The key innovation lies in dividing the network's parameters into two groups:
- Long-Term Memory (LTM) Units: These parameters are crucial for retaining knowledge of previously learned tasks. Their updates are restricted to prevent significant changes.
- Short-Term Memory (STM) Units: These parameters are more flexible and are allowed to adapt more readily to new tasks.
The Kalman Optimiser uses the Fisher Information Matrix to identify which parameters are important for previously learned tasks. This matrix, an approximation of the second-order derivatives of the loss function, helps determine the "importance" of each parameter. The authors then normalize this information to create a threshold for distinguishing LTM from STM units.
A Kalman Filter is then employed to update the weights. The filter uses the prior knowledge (weights and uncertainties from previous tasks) and the current task's data (represented by gradients as an uncertainty measure) to predict new weight values that minimize uncertainty. The LTM units are constrained by the Kalman update, ensuring they don't deviate too much from their learned values, while STM units are free to adapt more.
Datasets
The authors evaluated their method on the following standard datasets for continual learning:
- MNIST: Split into disjoint subsets of consecutive digits.
- CIFAR10: Split into two subsets of consecutive classes.
- CIFAR100: Used in a sequential learning scenario, starting with CIFAR10 and then introducing subsets of CIFAR100 classes.
Key Insights and Ideas
- Parameter Specialization: The idea that different parts of a neural network can be responsible for different tasks is a powerful concept. By explicitly separating parameters into LTM and STM, the model can achieve a more nuanced form of learning.
- Gradient as Uncertainty: Using the gradient of the loss function as a measure of parameter uncertainty is an intuitive way to quantify how "unsure" the model is about a particular parameter's current value, especially when encountering new data.
- Kalman Filter for Stability: The Kalman Filter's ability to incorporate prior knowledge and predict future states makes it a natural fit for managing parameter updates in a way that balances adaptation with stability.
- Fisher Information for Importance: Leveraging the Fisher Information Matrix provides a principled way to identify which parameters are most critical for preserving learned knowledge.
Key Results
The paper demonstrates that their proposed Kalman Optimiser significantly mitigates catastrophic forgetting.
- MNIST: The proposed method maintained high accuracy on previously learned tasks, while a baseline neural network showed a significant drop in performance. The average accuracy across all tasks also showed a steady increase with the Kalman Optimiser.
- CIFAR10/CIFAR100: Similar to MNIST, the Kalman Optimiser successfully retained performance on earlier tasks when learning new ones. The baseline model's performance on learned tasks degraded, whereas the proposed method showed a marked improvement in preserving knowledge.
Limitations and Future Work
- Computational Complexity: While the paper claims to address complexity issues compared to some other methods (e.g., using only one Fisher Information Matrix), computing and storing Fisher Information can still be computationally intensive, especially for very large networks.
- Threshold Selection: The paper mentions using a threshold to distinguish LTM and STM units, but the optimal way to set this threshold and its impact on performance might require further investigation.
- Dynamic Parameter Allocation: The paper states that parameters are dynamically decided at the end of each training process. Future work could explore more adaptive or online methods for re-allocating parameters between LTM and STM.
- Real-world Scenarios: While evaluated on standard benchmarks, extending the method to more complex and diverse real-world scenarios with varying task complexities and data distributions would be a valuable next step.
Summary updated