Generalized Variational Continual Learning

Noel Loo, Siddharth Swaroop, Richard E Turner

The Continual Learning Challenge

In Continual Learning (CL), an AI model is tasked with learning a sequence of different tasks one after another. The ultimate goal is to acquire new information without suffering from "catastrophic forgetting"—the tendency for new knowledge to overwrite previously learned knowledge.

Current approaches generally fall into two distinct camps:

  1. Variational methods (e.g., Variational Continual Learning or VCL): These treat network weights as probability distributions. While theoretically elegant, they often suffer from "overpruning." In this pathology, the model's internal neurons are permanently shut off after the first few tasks, wasting the network's capacity for future learning.
  2. Laplace-based methods (e.g., Online EWC): These rely on point estimates. They tend to be more stable in practice but lack the rich probabilistic benefits and uncertainty quantification of variational inference.

This paper introduces Generalized Variational Continual Learning (GVCL), a framework that unifies these two popular approaches. Furthermore, by integrating task-specific FiLM layers, the authors solve the "dead neuron" pathology inherent in variational methods, allowing models to maintain high accuracy and superior uncertainty calibration across long sequences of tasks.

Methods: Unification and Architectural Synergy

The paper makes two primary technical contributions that work in tandem to improve continual learning.

1. The GVCL Framework

The authors utilize "likelihood tempering"—adding a hyperparameter $\beta$ to the objective function—to create a unified algorithm. They mathematically prove that VCL is a specific case where $\beta=1$. Conversely, as $\beta$ approaches 0, the algorithm recovers Online EWC. This theoretical unification allows practitioners to interpolate between the two methods to find the optimal "sweet spot" for a given dataset.

2. Task-Specific FiLM Layers

To combat the overpruning problem, the authors introduce Feature-wise Linear Modulation (FiLM) layers. These act as tiny, task-specific "knobs" that scale and shift the activations of the shared neural network.

In standard VCL, if a neuron isn't needed for a specific task, the model shuts it down by setting its bias to a large negative value. Because this bias has "low variance," it becomes permanently stuck, and the neuron can never be used again for future tasks.

By introducing FiLM layers, the task-specific parameters handle the pruning for that specific task. The shared weights can then revert to their "prior" state without being permanently disabled, effectively freeing up the neuron for future tasks.

Filter utilization across tasks for GVCL vs. GVCL+FiLM.

Key Results and Insights

The authors evaluated GVCL on a wide range of benchmarks, including Split-MNIST, Split-CIFAR, a massive 8-dataset Mixed Vision Benchmark, and CHASY (a new benchmark introduced in this paper based on handwritten LaTeX symbols, designed to test the small-data regime).

Average accuracy across tasks for Easy-CHASY, Hard-CHASY, and Split-MNIST.
  • Superior Accuracy in Small-Data Regimes: On the new CHASY benchmark, GVCL with FiLM (GVCL-F) achieved an impressive 90.9% accuracy, significantly outperforming the previous state-of-the-art (HAT at 82.6%).
  • Competitive at Scale: On the large-scale Mixed Vision benchmark, GVCL-F matched the performance of strong baselines like HAT (approximately 80% accuracy) but offered a major advantage in uncertainty estimation.
  • Better Calibration: Unlike many CL methods that remain "overconfident" even when wrong, GVCL-F provides vastly superior calibration. Its Expected Calibration Error (ECE) was 0.32%, compared to 1.69% for HAT. This means the model accurately "knows what it doesn't know," a vital characteristic for safe real-world deployment.

Why Variational Models Fail in CL

The paper provides a deep dive into why variational models traditionally fail in continual learning. It is not merely that they forget; rather, they are "too aggressive" at shutting down parts of the network to satisfy the mathematical constraints of the prior. This leads to a "frozen" model that cannot absorb new information. The core insight is that architectural changes (like FiLM) and regularization changes (like GVCL) should not be studied in isolation. FiLM layers provide the necessary "safety valve" that makes variational regularization viable in practice.

Limitations and Future Work

While GVCL represents a significant step forward, the authors note a few limitations:

  • Optimization Difficulty: While $\beta \to 0$ theoretically recovers EWC, optimizing the GVCL cost for very small $\beta$ values is numerically challenging and requires extensive training times.
  • Task ID Requirement: The current implementation of FiLM layers requires knowing the "Task ID" at both training and test time. Future work could explore inferring the task identity directly from the data.
  • Memory Replay: The current work focuses purely on regularization and architecture. Combining GVCL with "memory replay" (storing a small buffer of old examples) could yield even higher performance ceilings.

Summary updated

Abstract

Continual learning deals with training models on new tasks and datasets in an online fashion. One strand of research has used probabilistic regularization for continual learning, with two of the main approaches in this vein being Online Elastic Weight Consolidation (Online EWC) and Variational Continual Learning (VCL). VCL employs variational inference, which in other settings has been improved empirically by applying likelihood-tempering. We show that applying this modification to VCL recovers Online EWC as a limiting case, allowing for interpolation between the two approaches. We term the general algorithm Generalized VCL (GVCL). In order to mitigate the observed overpruning effect of VI, we take inspiration from a common multi-task architecture, neural networks with task-specific FiLM layers, and find that this addition leads to significant performance gains, specifically for variational methods. In the small-data regime, GVCL strongly outperforms existing baselines. In larger datasets, GVCL with FiLM layers outperforms or is competitive with existing baselines in terms of accuracy, whilst also providing significantly better calibration.
BibTeX
@inproceedings{loo2021generalized,
  author = {Noel Loo and Siddharth Swaroop and Richard E Turner},
  title = {{Generalized Variational Continual Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=_IM-AfFhna9},
}
Figure