Abstract
Continual learning requires models to learn new tasks sequentially without forgetting previous ones. Current probabilistic approaches, such as Variational Continual Learning (VCL) and Online Elastic Weight Consolidation (Online EWC), have largely existed as separate paradigms. We introduce Generalized Variational Continual Learning (GVCL), a framework that unifies these methods through likelihood-tempering. Furthermore, we address the "overpruning" pathology inherent in variational methods by incorporating task-specific Feature Linear Modulation (FiLM) layers. Our results demonstrate that GVCL with FiLM (GVCL-F) significantly outperforms existing baselines in small-data regimes and provides superior uncertainty calibration on larger benchmarks.
The Challenge of Continual Learning
Neural networks typically suffer from catastrophic forgetting: when trained on a new task, the weights required for previous tasks are overwritten. Regularization-based solutions often fall into two categories: Laplace-based methods like Online EWC, which use point estimates, and Variational methods like VCL, which treat weights as distributions.
While VCL is theoretically robust, it suffers from a "pathological" pruning effect. In an effort to minimize complexity (the KL-divergence term in the ELBO), the model aggressively shuts down neurons. Once a neuron's bias is set to a large negative value to "silence" it, it is effectively locked out of future learning, wasting the network's total capacity.
Methodology: GVCL and FiLM Layers
1. Unifying VCL and EWC
We show that by applying "likelihood-tempering"—introducing a hyperparameter $\beta$ to the Evidence Lower Bound (ELBO)—VCL recovers Online EWC as a limiting case when $\beta \to 0$. This allows for a unified algorithm, GVCL, which can interpolate between the global curvature of VCL and the local curvature of EWC to find an optimal balance for a given dataset.
2. Solving Overpruning with FiLM
To mitigate overpruning, we introduce task-specific FiLM layers. These layers apply a simple linear transformation (scaling and shifting) to the activations of the main network. Crucially, because these parameters are task-specific, they allow the model to "silence" a neuron for Task A while keeping it available to be "re-activated" for Task B. This architectural change prevents the permanent loss of model capacity seen in standard variational approaches.
Experimental Results
Small-Data Dominance: The CHASY Benchmark
We evaluated GVCL-F on CHASY, a benchmark based on LaTeX symbols designed to test transfer in small-data regimes (16 samples per class). In these few-shot scenarios, GVCL-F significantly outperformed strong baselines like HAT and PathNet. For example, on Easy-CHASY, GVCL-F achieved 90.9% accuracy, a substantial lead over HAT's 82.6%.
Calibration and Large-Scale Performance
On larger benchmarks like Mixed Vision (a sequence of 8 distinct datasets), GVCL-F matched the accuracy of state-of-the-art methods like HAT while providing vastly superior Expected Calibration Error (ECE). GVCL-F achieved an ECE of 0.3%, compared to 1.7% for HAT. This indicates that the model's confidence scores are highly reliable, a critical requirement for real-world deployment.
Conclusion
GVCL provides a theoretical bridge between two major strands of continual learning research. By combining this unified framework with FiLM layers, we solve the overpruning problem that has previously limited variational methods. The resulting approach is not only more accurate in data-scarce environments but also provides the well-calibrated uncertainty estimates necessary for robust Bayesian inference.
Summary updated