Abstract
Continual Learning (CL) aims to learn from a non-stationary data stream where the underlying distribution changes over time. While recent advances have produced efficient memory-free methods in the offline CL (offCL) setting, where tasks are known in advance and data can be revisited, online CL (onCL) remains dominated by memory-based approaches. The transition from offCL to onCL is challenging, as many offline methods rely on (1) prior knowledge of task boundaries and (2) sophisticated scheduling or optimization schemes, both of which are unavailable when data arrives sequentially and can be seen only once. In this paper, we investigate the adaptation of state-of-the-art memory-free offCL methods to the online setting. We first show that augmenting these methods with lightweight prototypes significantly improves performance, albeit at the cost of increased Gradient Imbalance, resulting in a biased learning towards earlier tasks. To address this issue, we introduce Fine-Grained Hypergradients, an online mechanism for rebalancing gradient updates during training. Our experiments demonstrate that the synergy between prototype memory and hypergradient reweighting substantially enhances the performance of memory-free methods in onCL and surpasses onCL baselines. Code will be released upon acceptance.
Introduction
Continual Learning (CL) seeks to enable models to learn from a non-stationary stream of data without "catastrophically forgetting" previous knowledge. Research typically bifurcates into offline CL (offCL), where data can be revisited and task boundaries are known, and online CL (onCL), where data arrives in a single pass and boundaries are often unknown. While offCL has benefited from efficient "memory-free" methods like prompt-tuning, onCL remains dominated by memory-based approaches that store raw images—a practice often restricted by privacy or hardware constraints. This paper bridges the gap by adapting powerful offCL methods to the onCL setting using lightweight prototypes and a novel Fine-Grained Hypergradient (FGH) mechanism.
Methods
The authors propose a two-pronged strategy to adapt state-of-the-art offline methods (e.g., L2P, DualPrompt, CODA) to the online, task-free setting:
- Prototypes as a Proxy for Memory: Instead of storing raw images, the model maintains a single "prototype" vector for each class (a running average of feature representations). A secondary loss term ($L_P$) recalibrates the final classification layer using these prototypes, acting as a lightweight, privacy-preserving memory.
- Fine-Grained Hypergradients (FGH): Online models suffer from Gradient Imbalance (GI), where learning becomes biased toward early classes. FGH addresses this and the difficulty of tuning learning rates (LR) by learning a unique weighting coefficient for every trainable parameter. By using the dot product of consecutive gradients, FGH automatically amplifies updates for new classes while dampening them for old ones to preserve stability.
Results and Discussion
The researchers evaluated their approach on CIFAR100, CUB, and ImageNet-R benchmarks in both "Clear" and "Si-Blurry" (Stochastic incremental Blurry) settings.
- The Prototype Paradox: While prototypes help the final layer remember, they exacerbate Gradient Imbalance by creating stronger gradients for early classes. FGH reverses this, identifying that newer tasks require higher gradients for rapid adaptation.
- Superior Performance: Combining FGH and Prototypes with methods like ConvPrompt or CODA consistently outperformed dedicated online baselines like MVP and oLoRA. In many scenarios, this "memory-free" approach outperformed Experience Replay (ER) with a 1,000-image buffer. For instance, on CIFAR100 (Si-Blurry), ConvPrompt + Ours achieved 86.34% Average Performance.
- LR Robustness: FGH makes models significantly more robust to the initial learning rate, a critical advantage in online settings where optimal LRs change constantly.
Conclusion
This work demonstrates that memory-free offline CL methods can be successfully transitioned to the online setting. By replacing raw image storage with class prototypes and managing gradient dynamics via FGH, the proposed framework achieves state-of-the-art results. While FGH introduces a new hyperparameter ($\gamma$) and requires storing previous gradients, its stability and performance gains make it a compelling solution for privacy-sensitive or resource-constrained continual learning. Future work may explore these dynamics in architectures without pre-trained backbones.
Summary updated