Adversarial Image Generation and Training for Deep Convolutional Neural Networks

Ronghua Shi, Hai Shu, Hongtu Zhu, Ziqi Chen

Abstract

Deep Neural Networks (DNNs) are highly susceptible to adversarial attacks, where subtle, often imperceptible perturbations can lead to incorrect classifications. This paper introduces mFI-PSO, a novel white-box adversarial framework that combines differential geometry with population-based optimization. By leveraging Manifold-based First-order Influence (mFI) and Particle Swarm Optimization (PSO), the method provides unprecedented control over attack parameters, including the number of perturbed pixels, target class, and specific misclassification confidence levels. Experiments on MNIST and CIFAR-10 demonstrate that mFI-PSO achieves high success rates and can be effectively used for adversarial fine-tuning to enhance model robustness.

Introduction

Despite their state-of-the-art performance, DNNs remain fragile. Traditional adversarial attacks like FGSM, PGD, or CW often lack fine-grained control, typically perturbing all pixels or struggling to hit precise confidence targets for misclassification. Furthermore, many rely on standard Euclidean gradients, which may not accurately reflect the intrinsic geometry of a model's decision boundaries.

mFI-PSO addresses these gaps by offering a flexible, customizable attack tool. It allows researchers to define specific security testing scenarios, such as "one-pixel" attacks or high-confidence targeted attacks, bridging the gap between "all-or-nothing" perturbations and precise model auditing.

The mFI-PSO Framework

The mFI-PSO method operates through a multi-stage pipeline designed for maximum flexibility. It first identifies vulnerable images and pixels before optimizing the exact perturbation values.

The mFI-PSO pipeline: from dataset selection to adversarial training.

Manifold-based First-order Influence (mFI)

Instead of using standard Jacobian norms, mFI-PSO employs a Riemannian manifold approach to calculate pixel influence. This measure identifies which pixels are "intrinsically" influential to the model's output, creating a "vulnerability map" of the image. This approach is invariant under certain model transformations, making it a more reliable indicator of true model weakness than extrinsic gradient measures.

Pixel-level mFI maps for an MNIST image across different target classes.

Particle Swarm Optimization (PSO)

Once vulnerable pixels are selected, the attack must determine the optimal perturbation values. Because users can specify a target misclassification probability (e.g., $P_{err} \geq 0.99$), the resulting loss function is often non-convex or non-differentiable. PSO—a gradient-free, evolutionary algorithm—is used to navigate this complex search space to find the minimal perturbation required to meet the user's constraints.

Experimental Results

The researchers evaluated mFI-PSO using ResNet32 on the MNIST and CIFAR-10 benchmarks. Key findings include:

  • High Success Rates: On identified "vulnerable" subsets, mFI-PSO achieved near 100% attack success, outperforming established methods like FGSM and PGD.
  • Constraint Handling: When the number of perturbed pixels was strictly limited, mFI-PSO maintained high efficacy while methods like JSMA and DeepFool saw significant performance degradation.
  • Targeting Efficiency: In CIFAR-10 tests, mFI-PSO outperformed the Carlini & Wagner (CW) attack by over 24% on specific vulnerable image subsets.
  • Adversarial Defense: By performing "adversarial fine-tuning" using mFI-PSO generated images, the authors significantly improved model robustness. On MNIST, the number of highly vulnerable images in the test set was reduced from 325 to 132.

Conclusion and Future Work

mFI-PSO represents a significant step toward more controllable and interpretable adversarial attacks. While highly effective, the use of PSO introduces higher computational costs compared to single-step gradient methods, making real-time generation on high-resolution datasets like ImageNet a challenge. Future research will focus on adapting these manifold-based measures for black-box settings and improving the scalability of the optimization process for larger image scales.

Summary updated

BibTeX
@inproceedings{shi2022adversarial,
  author = {Ronghua Shi and Hai Shu and Hongtu Zhu and Ziqi Chen},
  title = {{Adversarial Image Generation and Training for Deep Convolutional Neural Networks}},
  booktitle = {IJCNN},
  year = {2022},
  doi = {10.1109/ijcnn55064.2022.9892433},
}
Figure