QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attacks

Hassan Ali, Hammad Tariq, Muhammad Abdullah Hanif, Faiq Khalid, Semeen Rehman, Rehan Ahmed, Muhammad Shafique

Abstract Adversarial examples have emerged as a significant threat to machine learning algorithms, especially to the convolutional neural networks (CNNs). In this paper, we propose two quantization-based defense mechanisms, Constant Quantization (CQ) and Trainable Quantization (TQ), to increase the robustness of CNNs against adversarial examples. CQ quantizes input pixel intensities based on a "fixed" number of quantization levels, while in TQ, the quantization levels are "iteratively learned during the training phase", thereby providing a stronger defense mechanism. We apply the proposed techniques on undefended CNNs against different state-of-the-art adversarial attacks from the open-source Cleverhans library. The experimental results demonstrate 50%-96% and 10%-50% increase in the classification accuracy of the perturbed images generated from the MNIST and the CIFAR-10 datasets, respectively, on commonly used CNN architectures available in the Cleverhans library.

Introduction

Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial attacks, where imperceptible noise added to an input causes catastrophic misclassification. An attacker can use gradient descent to calculate specific, invisible perturbations to an input image that will confidently fool the network. While several defenses exist—such as adversarial training, gradient masking, and pre-processing filters—they are often computationally expensive or can be bypassed by adaptive attackers. QuSecNets addresses this by introducing a novel, differentiable quantization layer (QL) at the input of a CNN to neutralize adversarial noise or force it into the visible spectrum.

Proposed Defense: QuSecNets

The core idea of QuSecNets is to apply a quantization filter to input images, reducing the granularity of pixel intensities to disrupt the subtle perturbations used by attackers. The authors propose two variants:

  1. Constant Quantization (CQ): The number of quantization levels and their thresholds are fixed based on an initial error-resilience analysis of the dataset.
  2. Trainable Quantization (TQ): Instead of using fixed thresholds, the network learns them iteratively during the standard training phase, allowing the defense to adapt to the specific data distribution.
Overview of the QuSecNets defense mechanism and its impact on image classification.

A significant challenge in implementing a trainable quantization layer is that standard step functions are non-differentiable, which prevents the use of backpropagation during training. QuSecNets overcomes this by mathematically modeling the quantization layer as the average of multiple shifted sigmoid functions. This approximation makes the layer differentiable, enabling the network to optimize quantization thresholds alongside standard weights.

Architecture of the differentiable quantization layer (QL).

Experimental Evaluation

The defense was evaluated using the MNIST and CIFAR-10 datasets against state-of-the-art attacks, including FGSM, Carlini & Wagner (C&W), and JSMA, under both white-box and black-box threat models.

Key Insights

  • Differentiable Quantization is Powerful: By integrating a dynamic pre-processing filter directly into the network's learning loop, QuSecNets creates a robust defense that is harder for attackers to circumvent.
  • Forcing Noise into the Open: The quantization layer forces previously "invisible" adversarial noise to clump together, making it highly perceptible to human subjective evaluation. This serves as a qualitative indicator of an ongoing attack.
  • Granularity vs. Security: Counterintuitively, increasing the number of quantization levels can decrease security. More levels mean smaller intervals between thresholds, giving adversarial noise more granular room to propagate through the network.
  • Trainable beats Constant: TQ generally outperforms CQ because learning the thresholds allows the network to adapt to the specific data distribution, making it inherently more robust to unexpected noise.

Results and Discussion

QuSecNets significantly improved classification accuracy under attack across multiple scenarios:

  • MNIST: Accuracy increased by 50%–96%. For instance, under a white-box FGSM attack, an undefended network's accuracy plummeted to 1.48%, while QuSecNets (CQ) restored it to 97.51%.
  • CIFAR-10: Accuracy increased by 10%–50%. Against the C&W-L2 attack, where an undefended network had 0% accuracy, QuSecNets restored accuracy to 54%.

Limitations and Future Work

Despite its success, QuSecNets shows sensitivity to dataset complexity; performance on the more complex CIFAR-10 is lower than on MNIST because the quantization noise itself can degrade baseline accuracy on clean images. Additionally, the defense is less effective against targeted JSMA attacks, which use high-intensity localized spikes rather than distributed low-intensity noise. Future work may involve combining QuSecNets with other spatial pre-processing techniques, such as median filters or input drop-out, to catch these high-intensity pixel spikes before they reach the quantization layer.

Summary updated

Abstract

Adversarial examples have emerged as a significant threat to machine learning algorithms, especially to the convolutional neural networks (CNNs). In this paper, we propose two quantization-based defense mechanisms, Constant Quantization (CQ) and Trainable Quantization (TQ), to increase the robustness of CNNs against adversarial examples. CQ quantizes input pixel intensities based on a "fixed" number of quantization levels, while in TQ, the quantization levels are "iteratively learned during the training phase", thereby providing a stronger defense mechanism. We apply the proposed techniques on undefended CNNs against different state-of-the-art adversarial attacks from the open-source \textit{Cleverhans} library. The experimental results demonstrate 50%-96% and 10%-50% increase in the classification accuracy of the perturbed images generated from the MNIST and the CIFAR-10 datasets, respectively, on commonly used CNN (Conv2D(64, 8x8) - Conv2D(128, 6x6) - Conv2D(128, 5x5) - Dense(10) - Softmax()) available in \textit{Cleverhans} library.
BibTeX
@inproceedings{ali2019qusecnets,
  author = {Hassan Ali and Hammad Tariq and Muhammad Abdullah Hanif and Faiq Khalid and Semeen Rehman and Rehan Ahmed and Muhammad Shafique},
  title = {{QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attacks}},
  booktitle = {IOLTS},
  year = {2019},
  url = {https://openreview.net/forum?id=brghIm5RmDn},
}
Figure