Memorization Weights for Instance Reweighting in Adversarial Training

Jianfu Zhang, Yan Hong, Qibin ZhaoAAAI 2023 · OpenReview

Abstract

Adversarial training is an effective way to defend deep neural networks (DNN) against adversarial examples. However, there are atypical samples that are rare and hard to learn, or even hurt DNNs' generalization performance on test data. In this paper, we propose a novel algorithm to reweight the training samples based on self-supervised techniques to mitigate the negative effects of the atypical samples. Specifically, a memory bank is built to record the popular samples as prototypes and calculate the memorization weight for each sample, evaluating the "typicalness" of a sample. All the training samples are reweighted based on the proposed memorization weights to reduce the negative effects of atypical samples. Experimental results show the proposed method is flexible to boost state-of-the-art adversarial training methods, improving both robustness and standard accuracy of DNNs.

Introduction

Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial examples—input images with tiny, invisible perturbations that cause misclassification. While Adversarial Training (AT) is the gold standard for defense, it often suffers from a significant "generalization gap," where models perform well on training data but poorly on unseen adversarial attacks.

The authors argue that this gap is largely driven by the presence of "atypical" samples in training sets. These samples are rare, noisy, or otherwise "training-unfriendly." Standard AT treats all samples equally, forcing the model to memorize these outliers. This consumes model capacity and results in a convoluted decision boundary that generalizes poorly. Existing supervised reweighting schemes often fail because they rely on output logits, which can be manipulated by adaptive attacks.

Methodology: The MeoW Framework

To address these issues, we introduce MeoW (Memorization Weights), a method that leverages a self-supervised memory bank to identify and reweight samples based on their "typicalness."

1. Self-Supervised Memory Bank

MeoW uses a discrete codebook (similar to VQ-VAEs) to store prototypes of feature embeddings. As training progresses, the model learns to map common, robust features to these prototypes. The "typicalness" of any given sample is defined by the Euclidean distance between its feature embedding and the nearest neighbor in the codebook. A small distance indicates a typical sample, while a large distance identifies an atypical one.

2. Dual Weighting Strategy

Unlike previous methods, MeoW applies a differentiated weighting strategy:

  • Adversarial Samples ($w^-$): Weights are assigned to suppress atypical samples, preventing the model from overfitting to noisy or outlier adversarial perturbations.
  • Natural Samples ($w^+$): Weights are assigned to amplify atypical samples, ensuring the model does not ignore legitimate but rare "long-tail" data in the clean distribution.
Figure 1: Conceptual visualization of how reweighting atypical samples simplifies and smooths the decision boundary, leading to better robustness.

3. Selective Codebook Updates

To maintain efficiency and focus, the memory bank is primarily updated using embeddings from misclassified samples, ensuring the prototypes represent the most critical regions of the feature space.

Experimental Evaluation

We evaluated MeoW on standard benchmarks including CIFAR-10, CIFAR-100, and SVHN across various architectures like PreActResNet-18 and WideResNet.

MeoW acts as a "plug-and-play" booster for existing state-of-the-art AT methods such as TRADES and AWP. On CIFAR-10, adding MeoW to vanilla AT improved robust accuracy under PGD-20 attacks from 45.30% to 54.61%. When combined with TRADES, it improved performance against the rigorous AutoAttack from 53.00% to 55.60%.

Figure 6: t-SNE visualization of embeddings on the CIFAR-10 dataset, showing how memorization weights cluster according to the typicalness of samples across different classes.

Key Insights and Discussion

  • Robustness-Accuracy Synergy: Unlike many defenses that trade standard accuracy for robustness, MeoW improves both by smoothing the decision boundary and making it more representative of the underlying data distribution.
  • Computational Efficiency: MeoW is highly efficient, adding only ~0.01s of latency per training batch and requiring zero overhead during inference, as the codebook is not needed for prediction.
  • Stability: By basing weights on self-supervised feature-space distances rather than supervised logits, MeoW is significantly more resistant to adaptive attacks designed to "game" reweighting logic.

Conclusion

MeoW provides a robust and flexible framework for instance reweighting in adversarial training. By identifying typical samples through a self-supervised memory bank, it mitigates the negative impact of atypical outliers. Future work will explore the limits of this approach against novel adaptive attacks and further refine the codebook's ability to adapt to shifting data distributions during training.

Summary updated

Abstract

Adversarial training is an effective way to defend deep neural networks (DNN) against adversarial examples. However, there are atypical samples that are rare and hard to learn, or even hurt DNNs' generalization performance on test data. In this paper, we propose a novel algorithm to reweight the training samples based on self-supervised techniques to mitigate the negative effects of the atypical samples. Specifically, a memory bank is built to record the popular samples as prototypes and calculate the memorization weight for each sample, evaluating the "typicalness" of a sample. All the training samples are reweigthed based on the proposed memorization weights to reduce the negative effects of atypical samples. Experimental results show the proposed method is flexible to boost state-of-the-art adversarial training methods, improving both robustness and standard accuracy of DNNs.
BibTeX
@inproceedings{zhang2023memorization,
  author = {Jianfu Zhang and Yan Hong and Qibin Zhao},
  title = {{Memorization Weights for Instance Reweighting in Adversarial Training}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=QGFM74N118},
}
Figure