Abstract
Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks that would trigger misclassification of DNNs but may be imperceptible to human perception. Adversarial defense has been important ways to improve the robustness of DNNs. Existing attack methods often construct adversarial examples relying on some metrics like the $\ell_p$ distance to perturb samples. However, these metrics can be insufficient to conduct adversarial attacks due to their limited perturbations. In this paper, we propose a new internal Wasserstein distance (IWD) to capture the semantic similarity of two samples, and thus it helps to obtain larger perturbations than currently used metrics such as the $\ell_p$ distance. We then apply the internal Wasserstein distance to perform adversarial attack and defense. In particular, we develop a novel attack method relying on IWD to calculate the similarities between an image and its adversarial examples. In this way, we can generate diverse and semantically similar adversarial examples that are more difficult to defend by existing defense methods. Moreover, we devise a new defense method relying on IWD to learn robust models against unseen adversarial examples. We provide both thorough theoretical and empirical evidence to support our methods.
Introduction: The Limits of the Pixel Box
Deep Neural Networks (DNNs) are notoriously fragile. Small, intentionally crafted perturbations—adversarial attacks—can cause them to misclassify images with high confidence. Traditionally, researchers have used the $\ell_p$ distance (like $\ell_\infty$ or $\ell_2$) to ensure these perturbations are "small" enough to be invisible to humans.
However, $\ell_p$ metrics are often a poor proxy for human perception. They only allow for tiny changes to individual pixels, effectively confining the attacker to a narrow "pixel box." This constraint misses a vast world of semantically similar transformations—like slight rotations, translations, or changes in texture—that humans find natural but DNNs find confusing. Consequently, models trained to be robust against $\ell_p$ attacks remain wide open to "manifold" attacks that transform the image in more complex, realistic ways.
Methodology: Internal Wasserstein Distance (IWD)
The core innovation of this work is the Internal Wasserstein Distance (IWD). Instead of comparing image $A$ to image $B$ pixel-by-pixel, IWD treats each image as a collection (distribution) of local patches. It then calculates the "cost" of transforming the patch distribution of one image into another. This allows for large pixel-level changes (like moving a volcano in a landscape) as long as the overall "style" and "content" patches remain consistent.
The Attack: IWDA
Using a generative approach, a generator $g$ creates an adversarial image while a discriminator $f$ tries to distinguish between the patch distributions of the original and the adversarial image. The attack succeeds when the generator creates an image that fools the target classifier $h$ while minimizing the IWD, ensuring the adversarial image remains semantically similar to the original.
The Defense: IWDD
To defend against such attacks, the authors derive a theoretical upper bound for classification error based on IWD. They employ a minimax optimization strategy, training the model to minimize the worst-case loss generated by their IWD-based attack. This "adversarial training" on the manifold helps the model cover "blind spots" in the data distribution that standard $\ell_p$ training misses.
Experimental Evaluation and Results
The researchers evaluated their methods on standard benchmarks including CIFAR-10 and a subset of ImageNet (e.g., volcano, coral reef).
Key Insights
- Semantic vs. Pixel Similarity: IWD captures "semantic consistency." An image can be very far from the original in $\ell_p$ space but very close in IWD space, proving that the perturbation ball defined by IWD is significantly larger than that of $\ell_p$.
- Diversity and Realism: Unlike standard attacks that produce "static noise," IWDA generates multiple, diverse adversarial examples (e.g., different sizes or positions of the main object) that look like natural variations of the scene.
Performance Metrics
- Attack Success: On ImageNet, IWDA achieved a 100% attack success rate (ASR) against several models that were supposedly "robust" (like ResNet-101 trained with free-PGD), whereas standard attacks like FGSM only achieved ~18% ASR.
- Superior Defense: The IWDD defense method consistently outperformed standard adversarial training. For example, on tiny ImageNet using an Inception-v3 backbone, IWDD achieved 46.00% accuracy against PGD-10 attacks, compared to 28.40% for standard robust training.
Conclusion
While IWD-based methods are more computationally expensive than simple gradient-based $\ell_p$ attacks, they reveal a critical vulnerability in current DNNs. By moving beyond pixel-wise constraints, IWD provides a more robust framework for both evaluating and improving the resilience of AI models against real-world, semantic variations. Future work will focus on scaling these methods to full-scale, high-resolution environments and developing more powerful universal generators.
Summary updated