Abstract
Deep Neural Networks (DNNs) are susceptible to adversarial attacks, and traditional adversarial training is computationally prohibitive for many researchers. This work proposes a cost-effective defense using short-term network fine-tuning with localized, random data augmentation. By perturbing local image patches and selecting appropriate optimizers—specifically RMSProp—the proposed method achieves significant robustness against gradient-based attacks (FGSM, IFGSM, PGD) while maintaining or improving clean accuracy. The approach offers a 60x speedup over standard adversarial training while often outperforming it in robustness metrics.
Introduction
Deep Neural Networks (DNNs) have revolutionized computer vision but remain notoriously vulnerable to adversarial attacks—small, intentional perturbations that cause models to misclassify images. While "Adversarial Training" (AT), which involves training a model on adversarial examples from scratch, is a common defense, it is incredibly resource-intensive. This "resource barrier" often prevents academic institutions with limited compute power from developing robust models.
This paper addresses this challenge by investigating whether robust models can be achieved through a lightweight fine-tuning approach rather than training from scratch. The authors focus on targeting local regions of an image where neurons exhibit the highest sensitivity to perturbations.
Methodology
The proposed defense mechanism utilizes a randomized sampling-based data augmentation strategy applied during a brief fine-tuning phase.
Local Patch Modification and Stochastic Selection
Instead of altering the entire image, the method selects $w \times w$ patches (typically centered). This is based on the observation that individual neurons are often sensitive to specific local features. The researchers curate a pool of 10 operations ($K$), including rotation, translation, flipping, noise addition, and pixel masking (occlusion). For each image, a random number $\lambda$ is drawn from a uniform distribution $U(10)$ to select a transformation $q$ from the pool. This "intelligent randomness" helps break the model's over-reliance on specific training distributions.
Fine-tuning and Optimizer Analysis
The network is fine-tuned for a short period (approximately 20 minutes) using a combination of original clean images and these locally modified images. A critical component of this study is the comparison of different optimizers—specifically Adam and RMSProp—to determine their impact on the final robustness of the model.
Results and Analysis
The approach was evaluated on three benchmarks: Fashion-MNIST (F-MNIST), CIFAR-10, and Imagenette. The defense was tested against white-box attacks including the Fast Gradient Sign Method (FGSM), Iterative FGSM (IFGSM), and Projected Gradient Descent (PGD).
Optimizer Performance
A key finding is that the choice of optimizer during fine-tuning is critical. While Adam is a common default, RMSProp provided superior robustness on complex datasets like CIFAR-10 and Imagenette. For instance, on CIFAR-10, RMSProp reached 60.37% accuracy against PGD attacks, while Adam only reached 34.45%.
Efficiency and Robustness Gains
The proposed fine-tuning took only 20 minutes, compared to the 20 hours required for full adversarial training on a standard GPU (1080ti). Despite this 60x speedup, the results often surpassed full adversarial training. On CIFAR-10, the proposed method achieved 60.37% accuracy against PGD, significantly higher than the 37.93% achieved by standard PGD-based Adversarial Training. Furthermore, unlike many defenses that degrade clean accuracy, this method improved clean image performance by 3% on CIFAR-10.
Conclusion
This research demonstrates that localized, stochastic data augmentation combined with the right optimizer (RMSProp) can provide a highly efficient and effective defense against adversarial attacks. By focusing on local vulnerabilities and leveraging lightweight fine-tuning, robust DNNs can be developed without the massive computational overhead of traditional adversarial training. Future work may involve automating the selection of patch locations and transformations to further enhance the "intelligence" of the defense mechanism.
Summary updated