Gradient-free Continual Learning

Grzegorz RypescarXiv 2025 · DOI · arXiv

Abstract

Catastrophic forgetting in continual learning is often attributed to the lack of access to historical data. This paper proposes that the root cause is "gradient blindness"—the inability to compute gradients for missing data. We introduce EvoCL, a method that utilizes a gradient-free evolution strategy combined with an adapter network to optimize model parameters without requiring stored exemplars. EvoCL demonstrates significant performance gains on standard benchmarks, particularly on simpler datasets where it outperforms existing state-of-the-art methods.

Introduction: The Gradient Blindness Problem

In Continual Learning (CL), a model learns a sequence of tasks. The primary challenge is catastrophic forgetting, where learning a new Task B degrades performance on a previously learned Task A. Traditional approaches mitigate this by storing a subset of old data (exemplars) or applying regularization.

However, a fundamental bottleneck exists: Gradient Availability. Standard optimizers like Stochastic Gradient Descent (SGD) rely on backpropagation. Without the original data for Task A, the gradient $\nabla_{\theta}L_A$ cannot be computed. Consequently, the optimizer is "blind" to the requirements of the previous task and overwrites critical parameters.

Comparison of Gradient-based vs. Gradient-free Continual Learning. Gradient-based methods fail when gradients for old tasks are unavailable, whereas gradient-free methods can optimize towards the intersection of low-error regions.

Methodology: EvoCL

The proposed method, EvoCL, operates in the Exemplar-Free Class-Incremental Learning (EFCIL) setting. It avoids storing raw images by employing three key components:

  1. Latent Feature Memorization: Instead of images, the model stores class-specific prototypes in the latent feature space.
  2. Adapter Network: To bridge the gap between the frozen previous model and the current evolving model, an auxiliary "adapter" maps features between these spaces. This allows for an approximation of the old task's loss ($\hat{L}_A$) on current parameters.
  3. Gradient-Free Optimization: Since the approximated loss $\hat{L}_A$ is derived through an external adapter and may not support standard backpropagation, EvoCL uses an evolution strategy. This optimizer only requires the scalar value of the loss function, allowing it to navigate the parameter space toward a solution that satisfies both current and historical task constraints.

Experimental Results

EvoCL was evaluated against several baselines, including AdaGauss, FeCAM, and LwF, across three benchmarks: MNIST, FashionMNIST, and CIFAR100.

Performance comparison of EvoCL against state-of-the-art methods on MNIST, FashionMNIST, and CIFAR100 benchmarks.

On simpler datasets like MNIST and FashionMNIST, EvoCL achieved a substantial lead. For instance, on MNIST with 5 tasks ($T=5$), EvoCL reached 74.5% last accuracy ($A_{last}$), a 24.1% improvement over AdaGauss (50.4%). On FashionMNIST ($T=5$), it maintained a 10.7% lead.

However, performance on the more complex CIFAR100 dataset was mixed. While EvoCL showed competitive incremental accuracy ($A_{inc}$), its final accuracy ($A_{last}$) of 37.4% on 10 tasks trailed behind AdaGauss's 46.1%. This suggests that while gradient-free optimization is robust against forgetting, it may struggle with the high-dimensional complexity of natural images.

Discussion and Conclusion

The success of EvoCL highlights that forgetting is partly an optimization failure. By bypassing the need for explicit gradients, evolution strategies can effectively utilize "black-box" loss approximations.

Despite its strengths in stability, EvoCL faces challenges regarding computational cost and scalability. Evolution strategies typically require significantly more function evaluations than gradient-based methods. Future research should focus on refining the loss approximation accuracy and improving the efficiency of the gradient-free solver to handle larger-scale models and more diverse datasets.

Summary updated

BibTeX
@misc{rypesc2025gradient,
  author = {Grzegorz Rypesc},
  title = {{Gradient-free Continual Learning}},
  year = {2025},
  eprint = {2504.01219},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.01219},
}
Figure