Introduction
Unsupervised Continual Learning (UCL) represents a significant frontier for autonomous systems, enabling agents to learn from raw, unlabeled data streams. However, real-world data is typically non-i.i.d., characterized by temporal correlations where objects appear sequentially. Standard Self-Supervised Learning (SSL) methods, designed for i.i.d. data, suffer from catastrophic forgetting in these scenarios, often performing worse than randomly initialized models as they overfit to current temporal features.
The EVOLVE Framework
To address these challenges, we propose EVOLVE, a hybrid cloud-client framework that leverages diverse, pre-trained "expert" models to guide local learning.
Hybrid Architecture and Expert Aggregation
EVOLVE utilizes a small target model on the client device and a library of large, pre-trained expert models (e.g., Swin Transformers, ResNets) in the cloud. Knowledge transfer is achieved via the Expert Aggregation Loss ($\mathcal{L}_E$), which employs the Hilbert-Schmidt Independence Criterion (HSIC) to align the local model's representations with those of the experts.
Dynamic Weight Adjustment
Recognizing that expert relevance varies by context, EVOLVE implements a dynamic weighting mechanism. A Confidence Metric is calculated for each expert based on its predictive consistency across different views of the same data. These weights are updated using a moving average, allowing the system to adapt to environmental shifts without prematurely discarding potentially useful experts.
Experimental Evaluation
EVOLVE was evaluated on benchmarks including CIFAR-10, TinyImageNet, CORe50, and Stream-51, specifically focusing on sequential-imbalanced (Seq-imb) setups that mimic real-world distribution shifts.
Results and Discussion
EVOLVE consistently outperformed existing UCL methods across all benchmarks.
Key findings include:
- Accuracy Gains: EVOLVE improved linear evaluation accuracy by 6.1% to 53.7% over state-of-the-art methods. On the challenging Stream-51 dataset, EVOLVE achieved over 84% accuracy when paired with a BYOL backbone, while standard SSL methods struggled to exceed 30%.
- Stability: While competing methods often show "sawtooth" performance due to forgetting, EVOLVE maintains steady, monotonic improvement.
- Efficiency: The framework is practical for real-time use, with communication overhead under 1ms over 5G networks (roughly 720kB per round).
Conclusion and Future Work
EVOLVE effectively mitigates catastrophic forgetting in UCL by using pre-trained experts as visual anchors. Future work will address privacy through techniques like Fully Homomorphic Encryption (FHE) and explore methods to reduce dependency on high-speed communication and specific expert libraries.
Summary updated