Abstract
Continual Learning (CL) aims to learn from a non-stationary data stream where the underlying distribution changes over time. While recent advances have produced efficient memory-free methods in the offline CL (offCL) setting, where tasks are known in advance and data can be revisited, online CL (onCL) remains dominated by memory-based approaches. The transition from offCL to onCL is challenging, as many offline methods rely on (1) prior knowledge of task boundaries and (2) sophisticated scheduling or optimization schemes, both of which are unavailable when data arrives sequentially and can be seen only once. In this paper, we investigate the adaptation of state-of-the-art memory-free offCL methods to the online setting. We first show that augmenting these methods with lightweight prototypes significantly improves performance, albeit at the cost of increased Gradient Imbalance, resulting in a biased learning towards earlier tasks. To address this issue, we introduce Fine-Grained Hypergradients, an online mechanism for rebalancing gradient updates during training. Our experiments demonstrate that the synergy between prototype memory and hypergradient reweighting substantially enhances the performance of memory-free methods in onCL and surpasses onCL baselines. Code will be released upon acceptance.
Introduction
Continual Learning (CL) involves training models on a continuous stream of data without forgetting past knowledge. This field is generally split into two paradigms: Offline CL (offCL), where data can be revisited and task boundaries are known, and Online CL (onCL), a more realistic setting where data is seen only once and task boundaries are often blurry. Currently, the most effective onCL methods rely on memory buffers to replay past data, which raises privacy and hardware concerns. While memory-free methods exist for offCL, they often fail in onCL due to their reliance on task boundaries and precise learning rate scheduling. This paper bridges the gap by adapting state-of-the-art memory-free offCL methods to the strict onCL environment.
Methodology
To adapt existing prompt-based offCL methods (such as L2P, DualPrompt, and CODA) to the task-free online setting, we introduce two key mechanisms:
- Prototypes as a Proxy for Memory: Instead of storing raw images, the model maintains a running average (a "prototype") of features for each class. These lightweight prototypes recalibrate the final classification layer, acting as a privacy-preserving memory proxy.
- Fine-Grained Hypergradients (FGH): Adapting offCL methods to onCL often leads to Gradient Imbalance (GI), where the model updates too strongly based on older classes. FGH dynamically learns a specific gradient weighting coefficient for every trainable parameter by analyzing the dot product of consecutive gradients. This acts as an automatic, parameter-specific learning rate adapter that corrects GI and removes the need for offline schedulers.
Experimental Results
We evaluated our approach on CIFAR-100, CUB, and ImageNet-R datasets in both "Clear" and "Si-Blurry" (Stochastic incremental Blurry) boundary settings. Our results show that the synergy between prototypes and FGH is critical for performance.
Key Findings
- State-of-the-Art Performance: Combined with FGH and prototypes, offCL methods crushed existing onCL baselines. For instance, on CIFAR-100 in the Si-Blurry setting, CODA + FGH + Prototypes achieved 79.47% average accuracy, compared to 41.42% for the onCL baseline MVP.
- Beating Memory Buffers: Our memory-free approach frequently matched or surpassed Experience Replay (ER) baselines that were allowed to store 1,000 past samples.
- Robustness to Learning Rates: FGH makes models highly robust to sub-optimal initial learning rates, which is crucial for real-world deployment where hyperparameter grid searches are impossible.
Discussion and Limitations
While FGH significantly improves performance, it introduces a new hyperparameter (the hypergradient learning rate, $\gamma$). Setting this too high can lead to training instability. Furthermore, FGH relies on the presence of prototypes; without them, the "reverse gradient imbalance" created by FGH can lead to forgetting older classes. Despite these considerations, the synergy between these components provides a promising path for scalable, privacy-preserving continual learning that bridges the gap between offline and online research.
Summary updated