The Role of 'Sign' and 'Direction' of Gradient on the Performance of CNN

Akshay Agarwal, Richa Singh, Mayank VatsaWorkshops @ CVPR 2020 · OpenReview

Abstract

State-of-the-art deep learning models have achieved superlative performance across multiple computer vision applications such as object recognition, face recognition, and digits/character classification. Most of these models highly rely on the gradient information flows through the network for learning. By utilizing this gradient information, a simple gradient sign method based attack is developed to fool the deep learning models. However, the primary concern with this attack is the perceptibility of noise for large degradation in classification accuracy. This research address the question of whether an imperceptible gradient noise can be generated to fool the deep neural networks? For this, the role of sign function in the gradient attack is analyzed. The analysis shows that without-sign function, i.e. gradient magnitude, not only leads to a successful attack mechanism but the noise is also imperceptible to the human observer. Extensive quantitative experiments performed using two convolutional neural networks validate the above observation. For instance, AlexNet architecture yields 63.54% accuracy on the CIFAR-10 database which reduces to 0.0% and 26.39% when sign (i.e., perceptible) and without-sign (i.e., imperceptible) of the gradient is utilized, respectively. Further, the role of the direction of the gradient for image manipulation is studied. When an image is manipulated in the positive direction of the gradient, an adversarial image is generated. On the other hand, if the opposite direction of the gradient is utilized for image manipulation, it is observed that the classification error rate of the CNN model is reduced. On AlexNet, the error rate of 36.46% reduces to 4.29% when images of CIFAR-10 are manipulated in the negative direction of the gradient.

Introduction

Deep learning models are notoriously vulnerable to adversarial attacks, where tiny, calculated perturbations to an image cause the model to misclassify it. The most famous method, the Fast Gradient Sign Method (FGSM), uses the sign of the gradient to perturb pixels. However, FGSM often produces "blocky" noise that is easily spotted by humans as the attack strength increases. This paper explores the mechanics of adversarial attacks and defenses by dissecting the gradient information used to train and fool Convolutional Neural Networks (CNNs).

Methodology

The researchers decouple the "sign" and "direction" of gradients to investigate two primary modifications to the standard gradient approach:

  • Fast Gradient Magnitude (FGM) Attack: Unlike FGSM, which uses $sign(\nabla J)$ (effectively pushing every pixel to its maximum allowed change in a specific direction), FGM uses the raw gradient magnitude $\nabla J$. This preserves the structural information of the gradient (like edges) rather than treating it as uniform directional noise.
  • Cooperative Examples (Defense): Standard adversarial attacks add gradient noise to maximize loss. The authors propose a defense where they subtract gradient noise from images the network initially got wrong ("negative examples") to minimize loss. They term these "Defended" versions (D-FGSM and D-FGM). This essentially "nudges" a difficult image into a region of the data manifold where the CNN can recognize it correctly.
Comparison of the raw gradient magnitude versus the sign of the gradient. The sign function produces high-frequency, blocky patterns that are easily detected by humans.

Experimental Results

The authors performed extensive quantitative experiments using two architectures (AlexNet and ResNet-18) across five diverse datasets: MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, and SVHN.

The "Sign" as the Source of Perceptibility

The analysis reveals that the sign() function is the culprit behind visible adversarial noise. By forcing perturbations to a fixed step size, it creates high-frequency patterns that the human eye detects as "grain" or "blocks." Raw magnitude (FGM) concentrates noise on edges and important features, making it nearly invisible even at high intensities. On CIFAR-10 using AlexNet, accuracy dropped from 63.54% to 0% using FGSM (perceptible), but still dropped significantly to 26.39% using FGM (imperceptible).

Gradient Direction as a Correction Vector

The gradient points toward the "steepest ascent" of the loss function. By moving in the exact opposite direction, the authors demonstrate that we can systematically correct "natural" errors made by the model. The "Cooperative" approach showed massive gains in rescuing misclassified images. On CIFAR-10 (AlexNet), the natural error rate of 36.46% was reduced to just 4.29% by manipulating the images in the negative gradient direction.

Visual comparison of adversarial examples (a) and defended cooperative examples (b).

Conclusion

This work demonstrates that gradient magnitude provides a stealthier attack vector than the gradient sign, while the direction of the gradient can be effectively used to correct model errors. While the proposed defense mechanism currently requires knowing the true label to calculate the gradient, the next step is finding ways to make CNNs inherently "prefer" those regions of the data manifold without needing manual image manipulation.

Summary updated

Abstract

State-of-the-art deep learning models have achieved superlative performance across multiple computer vision applications such as object recognition, face recognition, and digits/character classification. Most of these models highly rely on the gradient information flows through the network for learning. By utilizing this gradient information, a simple gradient sign method based attack is developed to fool the deep learning models. However, the primary concern with this attack is the perceptibility of noise for large degradation in classification accuracy. This research address the question of whether an imperceptible gradient noise can be generated to fool the deep neural networks? For this, the role of sign function in the gradient attack is analyzed. The analysis shows that without-sign function, i.e. gradient magnitude, not only leads to a successful attack mechanism but the noise is also imperceptible to the human observer. Extensive quantitative experiments performed using two convolutional neural networks validate the above observation. For instance, AlexNet architecture yields 63.54% accuracy on the CIFAR-10 database which reduces to 0.0% and 26.39% when sign (i.e., perceptible) and without-sign (i.e., imperceptible) of the gradient is utilized, respectively. Further, the role of the direction of the gradient for image manipulation is studied. When an image is manipulated in the positive direction of the gradient, an adversarial image is generated. On the other hand, if the opposite direction of the gradient is utilized for image manipulation, it is observed that the classification error rate of the CNN model is reduced. On AlexNet, the error rate of 36.46% reduces to 4.29% when images of CIFAR-10 are manipulated in the negative direction of the gradient.
BibTeX
@inproceedings{agarwal2020role,
  author = {Akshay Agarwal and Richa Singh and Mayank Vatsa},
  title = {{The Role of 'Sign' and 'Direction' of Gradient on the Performance of CNN}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=y7RLjynkuVva},
}
Figure