Harmonic Adversarial Attack Method

Wen Heng, Shuchang Zhou, Tingting JiangarXiv 2018 · arXiv

Introduction

Deep Neural Networks (DNNs) have demonstrated remarkable performance across various computer vision tasks, yet they remain susceptible to adversarial attacks. Traditional adversarial methods, such as the Fast Gradient Sign Method (FGSM) or Basic Iterative Method (BIM), typically introduce high-frequency pixel-wise noise. While effective at fooling models, these perturbations are often detectable by the human visual system, which is highly specialized for edge extraction.

The Harmonic Adversarial Attack Method (HAAM) addresses this limitation by generating "edge-free" perturbations using harmonic functions. These perturbations mimic natural lighting variations, shadows, and color shifts, making them nearly imperceptible to humans while maintaining high effectiveness and transferability across different architectures.

Methodology

The core of HAAM lies in the application of harmonic functions to create smooth perturbations. Mathematically, a harmonic function $f$ satisfies Laplace’s equation ($\Delta f = 0$). Since the Laplacian operator is a fundamental tool for edge detection, a perturbation satisfying this constraint is inherently "edge-free."

To construct these functions, the authors utilize analytic complex functions, such as polynomials and trigonometric functions, whose real and imaginary components are guaranteed to be harmonic. The method further enhances flexibility through linear combinations of harmonic base functions and affine transformations (rotation, scaling, and translation). The parameters of these functions are optimized via backpropagation to minimize model confidence in the correct class label.

Comparison of Harmonic and Noise Perturbations: (a) True label: bee, (b) Predicted label: damselfly (HAAM), (c) Harmonic perturbations, (d) Predicted label: necklace (Noise), (e) Noise perturbations.

Experimental Setup

The effectiveness of HAAM was evaluated using the ImageNet dataset, specifically the subset from the NIPS 2017 "Defense Against Adversarial Attack" competition, comprising a 1,000-image development set and a 5,000-image test set. The attacks were tested against several prominent architectures: ResNet50, SqueezeNet, VGG16, and DenseNet121.

To quantify "stealthiness," the authors introduced a new metric, Edge-SSIM (ESSIM). Unlike standard SSIM, ESSIM calculates similarity on the Laplacian (edge) maps of images, highlighting how well an attack preserves the original edge structure.

Results and Discussion

Visual Quality and Human Perception

Subjective human trials revealed a strong preference for HAAM-generated images over traditional noisy attacks, with preference rates ranging from 96% to 99%. HAAM consistently outperformed other methods on the ESSIM metric, maintaining high scores (near 0.9) even at large perturbation magnitudes, whereas noisy attacks showed significant degradation.

Transferability

HAAM demonstrates superior transferability in black-box scenarios. Because the perturbations are smooth and global rather than model-specific noise, they capture broader structural vulnerabilities. For instance, when generating attacks on ResNet50 and testing on VGG16, HAAM-c (the color variant) achieved transfer rates of approximately 43-47%, nearly doubling the performance of competing high-quality attack methods at similar SSIM levels.

Natural Appearance

A significant insight is that harmonic perturbations often resemble real-world photographic effects like uneven exposure or lens flares. This suggests that adversarial vulnerabilities may exist in the wild under common lighting conditions.

Conclusion

HAAM provides a novel framework for generating adversarial examples that are both effective against DNNs and stealthy to human observers. While the method can produce visible patterns on very flat backgrounds, it offers a significant advancement in balancing fooling rate with visual naturalness. Future work may involve refining these functions to handle flat regions better and utilizing HAAM for data augmentation to improve model robustness against natural lighting variations.

Summary updated

Abstract

Adversarial attacks find perturbations that can fool models into misclassifying images. Previous works had successes in generating noisy/edge-rich adversarial perturbations, at the cost of degradation of image quality. Such perturbations, even when they are small in scale, are usually easily spottable by human vision. In contrast, we propose Harmonic Adversar- ial Attack Methods (HAAM), that generates edge-free perturbations by using harmonic functions. The property of edge-free guarantees that the generated adversarial images can still preserve visual quality, even when perturbations are of large magnitudes. Experiments also show that adversaries generated by HAAM often have higher rates of success when transferring between models. In addition, we find harmonic perturbations can simulate natural phenomena like natural lighting and shadows. It would then be possible to help find corner cases for given models, as a first step to improving them.
BibTeX
@misc{heng2018harmonic,
  author = {Wen Heng and Shuchang Zhou and Tingting Jiang},
  title = {{Harmonic Adversarial Attack Method}},
  year = {2018},
  eprint = {1807.10590},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.10590},
}
Figure