Data-dependent and Oracle Bounds on Forgetting in Continual Learning

Lior Friedman, Ron Meir

Abstract

In continual learning, knowledge must be preserved and re-used between tasks, maintaining good transfer to future tasks and minimizing forgetting of previously learned ones. While several practical algorithms have been devised for this setting, there have been few theoretical works aiming to quantify and bound the degree of Forgetting in general settings. We provide both data-dependent and oracle upper bounds that apply regardless of model and algorithm choice, as well as bounds for Gibbs posteriors. We derive an algorithm inspired by our bounds and demonstrate empirically that our approach yields improved forward and backward transfer.

The Theoretical Gap in Continual Learning

Continual Learning (CL) involves an agent learning a sequence of tasks sequentially. A primary challenge is Catastrophic Forgetting, where learning new information causes the model to lose performance on previously mastered tasks. While numerous heuristics—such as experience replay or weight regularization—have been developed to mitigate this, a rigorous theoretical understanding has remained elusive.

Existing theoretical analyses are often constrained to specific regimes, such as linear regression or the Neural Tangent Kernel (NTK) limit. These constraints limit their applicability to the deep, non-linear architectures used in practice. This paper addresses this gap by introducing a model-agnostic framework based on PAC-Bayes theory.

Comparison of theoretical frameworks for continual learning.

As shown in the table above, while previous works were limited by architecture or task construction, the proposed PAC-Bayes approach provides bounds that are applicable to any architecture and a wide range of task settings.

A PAC-Bayesian Framework for Forgetting

The authors utilize the PAC-Bayes framework to derive "probably approximately correct" bounds on generalization error. This approach is particularly suited for CL because it naturally handles the transition from a prior distribution (knowledge from previous tasks) to a posterior distribution (knowledge updated with the current task).

Data-Dependent Bounds

The core contribution is a bound for Backward Transfer (BWT), which quantifies performance degradation on old tasks. This bound is composed of three intuitive terms:

  1. Empirical Loss: The model's performance on the current task.
  2. KL-Divergence: A measure of how much the model parameters have shifted from the previous task's state.
  3. Disagreement Term: A term capturing the "loss covariance" or landscape similarity between the old and new tasks.

Oracle Bounds and Gibbs Posteriors

For idealized learners using a Gibbs posterior, the authors derive "oracle" bounds. These represent the theoretical limit of performance if the underlying data distributions were known, providing a benchmark for practical algorithms.

From Theory to Practice: PAC-Bayes CL

The theoretical bounds directly inspire a new optimization objective. The proposed PAC-Bayes Continual Learning algorithm (implemented via Variational Inference, or VI) treats the posterior from task $t-1$ as the prior for task $t$. The objective function explicitly balances minimizing the current task's loss with a KL-regularization term that prevents the model from moving too far from the previous solution, thereby preserving old knowledge.

Empirical Results and Bound Tightness

The framework was evaluated on synthetic Gaussian data with varying task shifts (Similar, Gradual, and Orthogonal) and standard benchmarks like Permuted-MNIST and Split-CIFAR10.

Empirical Backward Transfer (BWT) loss vs. theoretical bounds for gradual and orthogonal task shifts.

A key finding is that the derived PAC-Bayes bounds are remarkably tight. Unlike many theoretical bounds that provide non-informative (vacuous) values, these bounds closely track the actual forgetting observed during training. In synthetic experiments, the VI algorithm often outperformed Elastic Weight Consolidation (EWC), particularly in maintaining backward transfer as the number of tasks increased.

Conclusion and Future Directions

This work provides the first general, model-agnostic theoretical foundation for understanding forgetting in continual learning. By demonstrating that PAC-Bayes bounds can be both informative and actionable, it opens new avenues for theoretically-grounded CL research.

Future work includes extending these proofs to non-independent task sequences (curriculum learning) and addressing the computational scaling of KL-divergence estimation for very large-scale neural networks.

Summary updated

Abstract

In continual learning, knowledge must be preserved and re-used between tasks, maintaining good transfer to future tasks and minimizing forgetting of previously learned ones. While several practical algorithms have been devised for this setting, there have been few theoretical works aiming to quantify and bound the degree of Forgetting in general settings. We provide both data-dependent and oracle upper bounds that apply regardless of model and algorithm choice, as well as bounds for Gibbs posteriors. We derive an algorithm inspired by our bounds and demonstrate empirically that our approach yields improved forward and backward transfer.
BibTeX
@misc{friedman2024data,
  author = {Lior Friedman and Ron Meir},
  title = {{Data-dependent and Oracle Bounds on Forgetting in Continual Learning}},
  year = {2024},
  eprint = {2406.09370},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2406.09370},
}
Figure