Mathematics of Continual Learning

Liangzu Peng, René VidalarXiv 2025 · DOI · arXiv

Introduction

Continual Learning (CL) is the ability of a machine learning model to learn a sequence of tasks without forgetting previously acquired knowledge. While modern deep learning has approached this problem through various empirical heuristics—such as experience replay or weight regularization—the field has often lacked a unified mathematical foundation. This paper bridges that gap by demonstrating that the roots of CL lie in the well-established field of Adaptive Filtering (AF) from the 1960s.

The Problem of Catastrophic Forgetting

Standard neural networks suffer from catastrophic forgetting: when trained on a new task, the gradient updates necessary to minimize the new loss often overwrite the weights critical for previous tasks. While the deep learning community has developed various "tricks" to mitigate this, the authors argue that signal processing algorithms like Least Mean Squares (LMS) and Kalman Filters already provide the rigorous mathematical framework needed to understand and solve these issues.

A Rosetta Stone for Continual Learning

The core contribution of this work is the establishment of a "Rosetta Stone" that maps classical signal processing algorithms to modern CL strategies. By categorizing CL methods into four primary families, the authors find their direct Adaptive Filtering counterparts.

Table 1: Mapping Adaptive Filtering algorithms to Continual Learning perspectives and methods.

As shown in the table above, the mapping includes:

  1. Least Mean Squares (LMS) ↔ Memoryless SGD: Basic online gradient descent is essentially the LMS algorithm, which follows the "minimal disturbance principle."
  2. Affine Projection Algorithm (APA) ↔ Replay-based Methods: APA uses a small buffer of recent samples to project the model onto a constraint set, mathematically mirroring modern constrained optimization in CL.
  3. Recursive Least-Squares (RLS) ↔ Regularization: RLS uses a "forgetting factor" to weight recent data, mapping directly to layer-wise regularization in deep networks.
  4. Kalman Filter (KF) ↔ Task Relationships: The KF provides a framework for modeling how solutions evolve, enabling Positive Backward Transfer where learning a new task improves performance on an old one.

Theoretical Insights: The Minimal Disturbance Principle

A central concept is the Minimal Disturbance Principle. This principle dictates that a model should be updated just enough to correct the error on the current sample while disturbing previous knowledge as minimally as possible.

Figure 1: Visualization of different learning paradigms within a task-error matrix, highlighting the focus areas of Finetuning, Online Optimization, and Continual Learning.

In Figure 1, we see how different paradigms interact with the error matrix $\epsilon_{ij}$ (representing error on task $i$ after learning task $j$). While standard finetuning focuses on the current task, Continual Learning aims to manage the entire column or specific off-diagonal elements to ensure stability across the sequence.

Key Results and Mathematical Guarantees

The paper provides several formal proofs that unify these disparate fields:

  • Equivalence of Frameworks: High-profile CL frameworks like Ideal Continual Learner (ICL) and Orthogonal Recursive Fitting (ORFit) are proven to be variations of the same projection-based mathematical operations.
  • Implicit Bias: Theorem 4 proves that replay-based methods like APA are implicitly biased toward finding "Minimum-Norm" solutions, explaining their superior generalization capabilities.
  • Non-Forgetting Guarantees: Theorem 6 provides a formal proof that Gradient Projection (GP) ensures output features remain invariant for past tasks even as weights evolve to accommodate new data.
  • Positive Backward Transfer: Using the Kalman Filter and Rauch-Tung-Striebel (RTS) smoothing, the authors provide a formal proof (Theorem 7) that learning a new task can mathematically improve performance on old ones if task relationships are modeled correctly.

Limitations and Future Directions

Despite these theoretical advances, challenges remain. Most proofs assume a "shared solution" exists for all tasks, which may not hold if tasks are contradictory. Additionally, second-order methods like RLS and Kalman Filters involve high computational complexity due to large matrix inversions. Future research may look toward subspace tracking, online dictionary learning, and kernel adaptive filtering to scale these mathematically grounded approaches to billion-parameter models.

Summary updated

BibTeX
@misc{peng2025mathematics,
  author = {Liangzu Peng and René Vidal},
  title = {{Mathematics of Continual Learning}},
  year = {2025},
  eprint = {2504.17963},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.17963},
}
Figure