Abstract
Deep learning models are notoriously vulnerable to adversarial perturbations and natural corruptions. Traditional Adversarial Training (AT) often results in a significant trade-off, reducing accuracy on clean images while failing to generalize to real-world distortions. This paper introduces Stylized Adversarial Training (SAT), a novel defense mechanism that utilizes style transfer to generate perceptually diverse adversarial examples. By forcing the model to resist perturbations that incorporate the style and content of different classes, SAT achieves state-of-the-art robustness against both white-box attacks and common corruptions without the typical drop in clean-image accuracy.
Introduction
Deep Learning models are fragile; invisible "adversarial" tweaks can trick a classifier into misidentifying a subject—for instance, seeing a wolf as a car. While training on these "broken" images is the standard defense, it typically introduces two major issues: a significant drop in clean-image accuracy and poor generalization to real-world "natural" corruptions like fog, snow, or motion blur.
The authors argue that existing defenses focus too narrowly on the classifier's decision boundary and ignore the richer perceptual information—the style and content—of the images themselves. SAT addresses this by generating adversaries that lie closer to the "natural image manifold," preventing the model from overfitting to non-natural mathematical artifacts.
Methodology
The core of the SAT framework consists of two main components: Stylized Adversary Generation and Max-Margin training.
Stylized Adversary Generation
Instead of adding simple gradient noise, SAT selects a "target" image from a different class and perturbs the original "source" image using a multi-task objective. This process minimizes three distinct losses:
- Style Loss ($L_s$): Aligns the source image's texture and style (calculated via Gram matrices) with the target image.
- Content Loss ($L_c$): Ensures the source image's high-level features remain consistent.
- Cross-entropy Loss ($L_{ce}$): Standard adversarial goal of pushing the image across the classification boundary.
Max-Margin Adversarial Training
Once stylized adversaries are generated, the model is trained using a contrastive-style loss designed to "stretch" the margins between classes in the feature space. The objective minimizes the distance between the clean image and its adversary while maximizing the distance between the adversary and the target class sample. This deep supervision is applied across multiple layers of the network to ensure robustness at every stage of feature extraction.
Experimental Evaluation
The authors evaluated SAT on several standard benchmarks, including CIFAR-10/100, SVHN, CINIC-10 (for distributional shift), and CIFAR-10-C (for common corruptions).
Key Results
- White-Box Attacks: On CIFAR-10, SAT achieved 53.9% accuracy against Auto-Attack, outperforming standard Adversarial Training and Feature Scattering methods.
- Natural Corruptions: SAT demonstrated a significant lead in handling real-world distortions. On the CIFAR-10-C benchmark, it achieved a mean accuracy of 89.0%, compared to 76.6% for the popular "Trades" defense.
- Clean Accuracy: Unlike standard PGD training, SAT maintains high clean-image accuracy (~91% on CIFAR-10), effectively breaking the traditional robustness-accuracy trade-off.
- Unconstrained Attacks: In tests against "occlusion" attacks, SAT was significantly more resilient, showing up to 620% better robustness than competing defenses in extreme cases.
Discussion and Conclusion
SAT demonstrates that incorporating perceptual diversity into adversarial training leads to more human-like, semantically meaningful features. While the approach is computationally intensive—taking roughly 32 hours to train on a Tesla V100 for the multi-step version—it offers a superior path toward models that are both accurate and robust. Future work may explore more sophisticated target selection or distillation techniques to further enhance efficiency and generalization.
Summary updated