Global Convergence of Continual Learning on Non-IID Data

Fei Zhu, Yujing Liu, Wenzhuo Liu, Zhaoxiang ZhangarXiv 2025 · DOI · arXiv

Abstract

Continual learning (CL) aims to enable models to learn tasks sequentially without suffering from catastrophic forgetting. Despite empirical progress, theoretical understanding has been limited by restrictive assumptions such as independent and identically distributed (i.i.d.) data and persistent excitation (PE). This paper provides the first comprehensive theoretical proof for the global convergence of CL under realistic, non-IID data conditions. By establishing almost sure convergence for both nonlinear regression and general linear models, the authors provide explicit rates for how models "forget" old tasks and accumulate "regret" on new ones, bridging the gap between theory and practice.

Introduction

The primary challenge in continual learning is catastrophic forgetting—the tendency of neural networks to lose previously acquired knowledge when learning new information. While various empirical strategies, such as experience replay or parameter isolation, have been developed to mitigate this, the mathematical foundations remain underdeveloped. Most existing theories rely on two "crutches": the i.i.d. assumption and the Persistent Excitation (PE) condition. The latter requires that data features are sufficiently diverse at every step to uniquely identify model parameters—a condition rarely met in real-world data streams. This paper seeks to provide a more robust theoretical framework that operates under significantly weaker conditions.

Theoretical Framework: Shared vs. Approximate Minimizers

The authors categorize CL problems into two distinct scenarios based on the relationship between task-specific optima.

Conceptual illustration of (a) Shared Global Minimizer and (b) Approximate Common Minimizer scenarios in continual learning.
  1. Shared Global Minimizer (Scenario a): In this setting, all tasks share at least one common "perfect" solution ($w^*$). This is applicable to certain nonlinear models like Logistic Regression and Stochastic Saturation models.
  2. Approximate Common Minimizer (Scenario b): This is a more realistic setting where tasks are slightly different, and their individual optima ($w_i^*$) do not perfectly overlap but instead revolve around a central ground truth $w^*$.

Proposed Methodology

To address these scenarios, the authors propose two novel algorithms:

  • Algorithm 1: Tailored for the Shared Global Minimizer case, it handles nonlinear models by analyzing the "energy" of estimation error over time.
  • Algorithm 2: Designed for the Approximate Common Minimizer case, it incorporates an additional adaptation gain sequence ($\beta_t$) to manage potentially unbounded feature data in linear models.

The convergence proofs utilize Stochastic Lyapunov functions and martingale estimation techniques. Unlike standard optimization proofs that often show convergence "in mean" (on average), these techniques allow the authors to prove "almost sure" convergence. This means that in any single execution of the learning process, the model is guaranteed to find the solution with probability 1.

Key Theoretical Insights

A major contribution of this work is the identification of the "Weakest Possible" Condition for convergence. The authors demonstrate that "persistent excitation" is not strictly necessary. Instead, convergence is guaranteed as long as the total information grows slightly faster than the logarithm of the number of tasks ($\log m = o(\lambda_{min}(m))$).

Furthermore, the paper provides explicit rates for learning metrics. For the shared minimizer case, the forgetting metric $F_t$ is shown to converge at a rate of $O(\sqrt{\frac{\log t}{t}})$, providing a mathematical bound on how quickly a model stabilizes across sequential tasks.

Results and Empirical Validation

While primarily theoretical, the paper includes synthetic numerical demonstrations to validate the findings. In a 2D parameter space with 100 tasks generated from three "meta-groups," the proposed regularized approach successfully identified the global minimizer. In contrast, standard Stochastic Gradient Descent (SGD) suffered from catastrophic forgetting, overfitting to the most recent task and losing the global perspective. The results showed that the proposed algorithms maintain optimal regret—the loss incurred on unseen tasks—even in noisy, non-IID environments.

Conclusion and Future Work

This research provides a rigorous foundation for understanding why and how continual learning can succeed without traditional data assumptions. By proving global convergence for nonlinear models and non-IID data, it moves the field closer to a unified theory of sequential learning. Future work will focus on extending the analysis of approximate minimizers to complex nonlinear settings, such as deep neural networks, and addressing the computational overhead associated with matrix inversions in high-dimensional spaces.

Summary updated

BibTeX
@misc{zhu2025global,
  author = {Fei Zhu and Yujing Liu and Wenzhuo Liu and Zhaoxiang Zhang},
  title = {{Global Convergence of Continual Learning on Non-IID Data}},
  year = {2025},
  eprint = {2503.18511},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.18511},
}
Figure