Improving Sustainability of Adversarial Examples in Class-Incremental Learning

Taifeng Liu, Xinjing Liu, Liangqiu Dong, Yang Liu, Yilong Yang, Zhuo Ma

TL;DR

Most adversarial attacks are designed for "static" AI models, but real-world systems often use Class-Incremental Learning (CIL) to learn new things over time. This paper introduces SAE (Sustainable Adversarial Examples), a novel framework that ensures adversarial perturbations remain effective even as a model’s decision boundaries shift and evolve, outperforming existing attacks by over 31% in long-term success rates.

What Problem Does This Paper Address?

Adversarial examples (AEs)—carefully crafted images that trick AI into misclassifying them—are usually "one-and-done." They are optimized for a specific version of a model. However, in many modern applications like autonomous driving or facial recognition, models undergo Class-Incremental Learning (CIL). They are sequentially updated with new classes and data.

When a model learns new classes, its internal "understanding" (the latent space) shifts. This is known as domain drift. Existing adversarial attacks often fail after just a few updates because the perturbations they rely on either become "benign noise" or lead to a different, unintended misclassification. The authors tackle the challenge of sustainability: how do we make an adversarial attack that survives through multiple rounds of model updates?

Methods

The authors propose the SAE framework, which focuses on making the "semantics" of the adversarial perturbation robust and generalized. It consists of two primary components:

  1. Semantic Correction Module (SCM): To prevent the attack from overfitting to the initial version of the model, SAE uses CLIP (a massive vision-language model) as a "semantic anchor." CLIP provides universal, stable representations of classes. The SCM aligns the adversarial image's features with the target class's text features in CLIP’s space, while simultaneously using the specific CIL model’s gradients to "correct" the direction, ensuring the attack is both generalized and effective against the specific architecture.
  2. Filtering-and-Augmentation Module (FAM): Not all images are good candidates for an attack. If a "bicycle" image naturally contains a "road" (the target class), the resulting perturbation might be unstable. FAM identifies and filters out these "confusing" examples by measuring cosine similarity in the latent space. The remaining examples are then subjected to various augmentations (rotation, scaling, etc.) to ensure the generated perturbation is robust to different visual perspectives.

Datasets

The researchers evaluated SAE using two standard benchmarks:

  • CIFAR-100: 100 classes, split into 10 incremental tasks.
  • ImageNet-100: A subset of the massive ImageNet dataset, also split into 10 incremental tasks.

To generate the attacks without knowing the model's training data, they used Public Out-Of-Distribution (POOD) datasets: Tiny-ImageNet (for CIFAR experiments) and ImageNet-1K (for ImageNet-100 experiments).

Key Insights and Ideas

  • Sustainability vs. Transferability: While many researchers focus on "transferability" (making an attack work on different model architectures), this paper highlights "sustainability" (making an attack work on the same model as it evolves over time).
  • The Power of Foundation Models: By using CLIP as a reference point, the authors found a way to ground adversarial perturbations in "universal" semantics that don't change, even when the specific CIL model's decision boundaries do.
  • Semantic Stability: The "aha moment" here is that AEs fail in CIL because their semantic features are too fragile. By filtering out images with overlapping class features and using heavy augmentation, the perturbation becomes a "stable" feature that the model continues to associate with the target class across many updates.

Key Results

The authors measured the Sustainable Attack Success Rate (SASR), which averages the attack's success across all stages of the incremental learning process.

  • Massive Improvement: SAE outperformed state-of-the-art baselines (like AIM and UnivIntruder) by an average of 31.28%.
  • Long-term Survival: On CIFAR-100, SAE maintained a high success rate even after the model had learned 90 additional classes. In contrast, traditional attacks like MIFGSM saw their success rates plummet to near zero after only 30-40 new classes were added.
  • Visual Evidence: GradCAM visualizations showed that SAE-perturbed images consistently forced the model to focus on the same regions to make its (incorrect) prediction, whereas other attacks saw the model's focus "shrink" or dissipate over time.

Limitations and Future Work

The authors acknowledge a few hurdles:

  • The "Cache" Wall: In CIL methods that use fixed-length caches to store old data (like iCaRL), SAE experiences a performance dip once the cache becomes full (usually around 70 classes). This suggests that the attack is still somewhat tied to how well the model remembers its early training.
  • Class Confusion: The attack is less effective when the target class has very "confusing" or abstract semantics (e.g., "cloud" or "shrew") compared to distinct objects like "bicycle."
  • Future Directions: The paper suggests exploring untargeted attacks in CIL settings and investigating how these sustainable AEs might be used to actually improve CIL model robustness through adversarial training.

Summary updated

Abstract

Current adversarial examples (AEs) are typically designed for static models. However, with the wide application of Class-Incremental Learning (CIL), models are no longer static and need to be updated with new data distributed and labeled differently from the old ones. As a result, existing AEs often fail after CIL updates due to significant domain drift. In this paper, we propose SAE to enhance the sustainability of AEs against CIL. The core idea of SAE is to enhance the robustness of AE semantics against domain drift by making them more similar to the target class while distinguishing them from all other classes. Achieving this is challenging, as relying solely on the initial CIL model to optimize AE semantics often leads to overfitting. To resolve the problem, we propose a Semantic Correction Module. This module encourages the AE semantics to be generalized, based on a visual-language model capable of producing universal semantics. Additionally, it incorporates the CIL model to correct the optimization direction of the AE semantics, guiding them closer to the target class. To further reduce fluctuations in AE semantics, we propose a Filtering-and-Augmentation Module, which first identifies non-target examples with target-class semantics in the latent space and then augments them to foster more stable semantics. Comprehensive experiments demonstrate that SAE outperforms baselines by an average of 31.28% when updated with a 9-fold increase in the number of classes.
BibTeX
@misc{liu2025improving,
  author = {Taifeng Liu and Xinjing Liu and Liangqiu Dong and Yang Liu and Yilong Yang and Zhuo Ma},
  title = {{Improving Sustainability of Adversarial Examples in Class-Incremental Learning}},
  year = {2025},
  eprint = {2511.09088},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.09088},
}
Figure