Salient Frequency-aware Exemplar Compression for Resource-constrained Online Continual Learning

Junsu Kim, Suhyun KimAAAI 2025 · DOI

TL;DR

This paper introduces Salient Frequency-aware Exemplar Compression (SFEC), a novel method for efficient image compression in Online Class-Incremental Learning (OCIL). SFEC addresses the challenge of catastrophic forgetting in resource-constrained environments by intelligently compressing training data, preserving crucial information while minimizing computational overhead and compression artifacts.

What Problem Does This Paper Address?

Online Class-Incremental Learning (OCIL) is crucial for updating AI models with streaming data without storing the entire dataset, especially on edge devices with limited memory and computational power. A major hurdle in OCIL is "catastrophic forgetting," where models lose previously learned knowledge when trained on new data. While exemplar replay (storing representative samples of old data) helps, limited buffer capacity restricts the quality of stored data. Image compression offers a solution to store more exemplars, but existing methods often introduce significant computational overhead or detrimental compression artifacts that hinder model training. This paper tackles the problem of achieving effective exemplar compression for OCIL that is both computationally efficient and preserves crucial information for learning, particularly in resource-constrained settings.

Methods

The core of the proposed solution is the Salient Frequency-aware Exemplar Compression (SFEC) framework. SFEC builds upon the JPEG compression standard for its computational efficiency, as it doesn't require additional forward/backward passes for compression. However, standard JPEG aggressively quantizes high-frequency components, which are vital for neural network learning. SFEC addresses this by:

  • Saliency-aware Frequency Quantization: Instead of uniformly quantizing high frequencies, SFEC identifies "salient" frequencies within an image (based on gradients of spatial frequencies) and quantizes less salient frequencies more aggressively. This preserves important information while still achieving compression.
  • Compression-Aware Buffer Management: To further mitigate compression artifacts, SFEC stores the reconstruction error between the original and compressed images. When the buffer is full, it uses this reconstruction error (as a measure of distortion) to prioritize the eviction of more distorted exemplars, ensuring the buffer retains higher-quality data for replay.

Datasets

The paper evaluates SFEC on two standard OCIL benchmark datasets:

  • CIFAR-100: A dataset of 100 classes of images.
  • Mini-ImageNet: A subset of ImageNet, also used for image classification tasks.

The datasets are split into ten tasks for incremental learning evaluation.

Key Insights and Ideas

  • JPEG's High-Frequency Blindness is a Problem for NNs: Standard JPEG compression, optimized for human vision, discards high-frequency information that neural networks rely on for learning. SFEC's approach of selectively quantizing based on saliency is a key insight to overcome this.
  • Computational Overhead is a Bottleneck for OCIL: Existing compression methods, while effective at storing more data, introduce significant computational delays that can cause the model to miss learning opportunities from the streaming data. SFEC's JPEG-based approach minimizes this overhead.
  • Distortion-Aware Replay is Crucial: Simply compressing data isn't enough; managing which compressed data is kept is vital. SFEC's buffer management, which prioritizes keeping less distorted exemplars, is a novel way to ensure replay quality.
  • SFEC is a Drop-in Enhancement: The framework is designed to be integrated with existing OCIL methods with minimal overhead, as demonstrated by its successful application to the PCR (Proxy-based Contrastive Replay) method.

Key Results

  • Fast Streaming Scenario: SFEC consistently outperforms the baseline ER-ACE by a significant margin (1.87% and 2.61% on CIFAR-100 and Mini-ImageNet, respectively). It also shows superior performance compared to other exemplar compression methods like naive JPEG, AQM, and MRDC.
  • Slow Streaming Scenario: SFEC demonstrates significant improvement over the baseline, effectively mitigating compression defects. While MRDC initially shows promise, its performance degrades due to JPEG artifacts, whereas SFEC maintains strong performance.
  • Computational Efficiency: SFEC has a computational overhead (GFLOPS) and delay (δ) comparable to the baseline ER-ACE, making it highly suitable for resource-constrained environments. Other methods like CIM and MRDC have significantly higher overheads and delays.
  • Hyperparameter Sensitivity: The proposed method shows consistent performance across different values of its hyperparameter (λ), indicating robustness and negating the need for extensive tuning.

Limitations and Future Work

  • Limitations: The paper acknowledges that while SFEC significantly reduces compression artifacts compared to naive JPEG, some compression defects may still exist. The effectiveness of the saliency-aware quantization is dependent on the accuracy of the saliency estimation.
  • Future Work: The authors suggest that SFEC can be integrated with other state-of-the-art OCIL frameworks to further enhance their performance. Exploring more advanced saliency detection mechanisms or adaptive quantization strategies could potentially lead to even better compression quality and learning performance. Further investigation into the trade-offs between compression ratio, computational cost, and learning performance for different types of data and tasks would also be valuable.

Summary updated

BibTeX
@inproceedings{kim2025salient,
  author = {Junsu Kim and Suhyun Kim},
  title = {{Salient Frequency-aware Exemplar Compression for Resource-constrained Online Continual Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i17.33968},
}
Figure