Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting

Ryo Karakida, Shotaro Akaho

In the world of deep learning, we often operate under the assumption that "more data is better." However, when we train models sequentially—learning one task after another—this intuition can break down. A theoretical study from researchers at AIST Japan provides a rigorous look at the "learning curves" of continual learning, revealing counterintuitive scenarios where models can actually "forget" their own knowledge.

By utilizing the Neural Tangent Kernel (NTK) framework, the authors predict exactly when sequential training helps or hurts a model's performance. Most notably, they identify a phenomenon called self-knowledge forgetting, where learning the exact same task twice with different sample sizes can increase generalization error, challenging the "more is always better" mantra.

The Challenge of Continual Learning

Continual learning (CL) is notorious for catastrophic forgetting, a scenario where a model loses its ability to perform an old task after learning a new one. While the community has developed numerous algorithms to mitigate this, we still lack a fundamental theoretical understanding of the underlying learning curves.

Specifically, it is often unclear under what exact conditions sequential training leads to positive transfer (improving on a new task thanks to old knowledge) versus negative transfer (performing worse than if the model had started from scratch). This paper moves beyond empirical trial-and-error, providing analytical tools to predict these outcomes based on task similarity and sample sizes.

Theoretical Framework: NTK and the Replica Method

The authors analyze neural networks in the Neural Tangent Kernel (NTK) regime. In this regime, infinitely wide neural networks behave similarly to linear models, making their dynamics mathematically solvable.

To derive their results, the researchers employ the replica method, a sophisticated tool borrowed from statistical mechanics. This allows them to calculate "typical-case" generalization errors—essentially predicting the average performance a model will achieve given a certain number of training samples ($N$) and a specific level of task similarity ($\rho$). They successfully extend existing single-task theories to handle sequential updates, where the final weights from Task A serve as the initialization for Task B.

Key Insights: Forgetting and Transfer

The theoretical analysis yields several striking insights into how neural networks process sequential information:

  • The Critical Similarity Threshold: There is a specific "critical value" for task similarity. If Task B is too different from Task A (falling below this threshold), training on Task A first actually makes the model worse at Task B than if it had simply been trained from scratch.
  • Forgetting is "Literally Catastrophic": The theory demonstrates that backward transfer (forgetting) is far more sensitive than forward transfer. Even a microscopic dissimilarity between tasks can cause a massive spike in the error for the previously learned task.
  • Self-Knowledge Transfer vs. Forgetting: This is the paper's most novel finding. Even if you train a model on the exact same task twice (using different data samples), performance does not always improve.
  • Transfer: If sample sizes are equal across tasks, the generalization error decreases monotonically.
  • Forgetting: If sample sizes are unbalanced (e.g., a large first task followed by a small second task), the model can actually "forget" the optimal solution it found during the first task, leading to a higher overall generalization error.
Theoretical learning curves showing catastrophic forgetting (left) and self-knowledge forgetting (right).

Empirical Validation

To ensure these theoretical findings hold up in practice, the authors conducted empirical "sanity checks" using standard datasets:

  • Synthetic Data: Inputs generated on a hyper-sphere allowed for precise control over task similarity ($\rho$).
  • MNIST & CIFAR-10: The authors trained a 4-layer Multi-Layer Perceptron (MLP) on MNIST and a ResNet-18 on CIFAR-10.

These experiments qualitatively matched the theoretical NTK curves. Specifically, they confirmed that unbalanced sample sizes in sequential training create the predicted "humps" of increased error, validating the existence of self-knowledge forgetting in finite-width, practical architectures.

Limitations and Future Directions

The primary limitation of this study is its reliance on the NTK regime. While NTK provides a powerful mathematical lens, it assumes the network is so wide that its internal features don't change significantly during training (i.e., no "feature learning"). Because real-world deep learning heavily relies on learning new representations, this theory may not capture the full picture of complex model dynamics.

Additionally, the study assumes the input distribution remains the same across tasks (no domain shift). In many real-world continual learning problems, the distribution of inputs changes over time. Extending this analytical framework to handle distribution shifts and complex algorithmic strategies, such as experience replay, represents a promising frontier for future research.

Summary updated

Abstract

Sequential training from task to task is becoming one of the major objects in deep learning applications such as continual learning and transfer learning. Nevertheless, it remains unclear under what conditions the trained model's performance improves or deteriorates. To deepen our understanding of sequential training, this study provides a theoretical analysis of generalization performance in a solvable case of continual learning. We consider neural networks in the neural tangent kernel (NTK) regime that continually learn target functions from task to task, and investigate the generalization by using an established statistical mechanical analysis of kernel ridge-less regression. We first show characteristic transitions from positive to negative transfer. More similar targets above a specific critical value can achieve positive knowledge transfer for the subsequent task while catastrophic forgetting occurs even with very similar targets. Next, we investigate a variant of continual learning which supposes the same target function in multiple tasks. Even for the same target, the trained model shows some transfer and forgetting depending on the sample size of each task. We can guarantee that the generalization error monotonically decreases from task to task for equal sample sizes while unbalanced sample sizes deteriorate the generalization. We respectively refer to these improvement and deterioration as self-knowledge transfer and forgetting, and empirically confirm them in realistic training of deep neural networks as well.
BibTeX
@inproceedings{karakida2022learning,
  author = {Ryo Karakida and Shotaro Akaho},
  title = {{Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=tFgdrQbbaa},
}
Figure