Adversarial Amendment is the Only Force Capable of Transforming an Enemy into a Friend

Chong Yu, Tao Chen, Zhongxue Gan

Abstract

Adversarial attack is commonly regarded as a huge threat to neural networks because of misleading behavior. This paper presents an opposite perspective: adversarial attacks can be harnessed to improve neural models if amended correctly. Unlike traditional adversarial defense or adversarial training schemes that aim to improve the adversarial robustness, the proposed adversarial amendment (AdvAmd) method aims to improve the original accuracy level of neural models on benign samples. We thoroughly analyze the distribution mismatch between the benign and adversarial samples. This distribution mismatch and the mutual learning mechanism with the same learning ratio applied in prior art defense strategies is the main cause leading the accuracy degradation for benign samples. The proposed AdvAmd is demonstrated to steadily heal the accuracy degradation and even leads to a certain accuracy boost of common neural models on benign classification, object detection, and segmentation tasks. The efficacy of the AdvAmd is contributed by three key components: mediate samples (to reduce the influence of distribution mismatch with a fine-grained amendment), auxiliary batch norm (to solve the mutual learning mechanism and the smoother judgment surface), and AdvAmd loss (to adjust the learning ratios according to different attack vulnerabilities) through quantitative and ablation experiments.

The Robustness-Accuracy Trade-off

In deep learning, a persistent challenge is the "robustness-accuracy trade-off." Standard adversarial training, designed to make models resistant to attacks, typically incurs an "accuracy tax"—a drop in performance on normal, benign data. This research identifies two primary causes for this degradation:

  1. Distribution Mismatch: Adversarial samples and benign samples exhibit different internal statistics, creating a gap that standard models struggle to bridge.
  2. Boundary Fuzziness: Traditional defense methods attempt "mutual learning," using the same parameters for both distributions. This results in a smoother but less precise decision boundary, leading to new errors on benign data.
Conceptual comparison of vanilla detection, adversarial attacks, and defense mechanisms illustrating distribution mismatch and boundary fuzziness.

The Adversarial Amendment (AdvAmd) Framework

Rather than treating adversarial attacks as enemies to be blocked, the authors propose Adversarial Amendment (AdvAmd). This framework treats adversarial examples as high-value "hard examples" that can refine decision boundaries. AdvAmd consists of three core components:

  • Mediate Samples: The method generates intermediate samples between benign and adversarial states. This provides a fine-grained "bridge" that helps the model navigate the distribution shift more effectively than training on final adversarial states alone.
  • Auxiliary Batch Norm (BN): To prevent adversarial statistics from "polluting" benign data, AdvAmd introduces a separate, auxiliary BN path specifically for adversarial samples. This disentanglement keeps the decision boundaries sharp for normal data.
  • AdvAmd Loss: Recognizing that different categories vary in vulnerability, the authors designed a loss function with an "attack vulnerability coefficient." This directs the model to prioritize learning from categories that are more susceptible to being fooled.

Quantitative Results and Performance Boost

The researchers evaluated AdvAmd across standard benchmarks including ImageNet and COCO, testing architectures like ResNet-50, Vision Transformers (DeiT), and various detection models (YOLO-v5, Faster R-CNN).

Delta Top-1 accuracy on benign ImageNet samples, showing AdvAmd as the only method providing a positive performance boost compared to traditional defenses.

Key findings include:

  • Benign Accuracy Boost: On the COCO dataset, AdvAmd achieved a 1.2% to 2.5% metric increase on benign samples compared to vanilla models.
  • Healing the Accuracy Tax: While traditional defenses like Adv-Train or MagNet caused noticeable accuracy degradation, AdvAmd consistently improved upon the baseline performance.
  • Cross-Task Versatility: The performance gains were consistent across classification, detection, and segmentation tasks, proving the method's robustness across different neural architectures.

Conclusion and Future Work

AdvAmd demonstrates that adversarial attacks, when properly amended, can serve as a powerful tool for model refinement rather than just a security threat. While the current framework focuses on category-based attacks, future work will explore extending AdvAmd to address spatial perturbations, such as those targeting object positioning or segmentation masks.

Summary updated

Abstract

Adversarial attack is commonly regarded as a huge threat to neural networks because of misleading behavior. This paper presents an opposite perspective: adversarial attacks can be harnessed to improve neural models if amended correctly. Unlike traditional adversarial defense or adversarial training schemes that aim to improve the adversarial robustness, the proposed adversarial amendment (AdvAmd) method aims to improve the original accuracy level of neural models on benign samples. We thoroughly analyze the distribution mismatch between the benign and adversarial samples. This distribution mismatch and the mutual learning mechanism with the same learning ratio applied in prior art defense strategies is the main cause leading the accuracy degradation for benign samples. The proposed AdvAmd is demonstrated to steadily heal the accuracy degradation and even leads to a certain accuracy boost of common neural models on benign classification, object detection, and segmentation tasks. The efficacy of the AdvAmd is contributed by three key components: mediate samples (to reduce the influence of distribution mismatch with a fine-grained amendment), auxiliary batch norm (to solve the mutual learning mechanism and the smoother judgment surface), and AdvAmd loss (to adjust the learning ratios according to different attack vulnerabilities) through quantitative and ablation experiments.
BibTeX
@inproceedings{yu2023adversarial,
  author = {Chong Yu and Tao Chen and Zhongxue Gan},
  title = {{Adversarial Amendment is the Only Force Capable of Transforming an Enemy into a Friend}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/503},
}
Figure