Abstract: Neural networks have achieved remarkable performance in computer vision, however they are vulnerable to adversarial examples. Adversarial examples are inputs that have been carefully perturbed to fool classifier networks, while appearing unchanged to humans. Based on prior works on detecting adversaries, we propose a structured methodology of augmenting a deep neural network (DNN) with a detector subnetwork. We use $\textit{Adversarial Noise Sensitivity}$ (ANS), a novel metric for measuring the adversarial gradient contribution of different intermediate layers of a network. Based on the ANS value, we append a detector to the most sensitive layer. In prior works, more complex detectors were added to a DNN, increasing the inference computational cost of the model. In contrast, our structured and strategic addition of a detector to a DNN reduces the complexity of the model while making the overall network adversarially resilient. Through comprehensive white-box and black-box experiments on MNIST, CIFAR-10, and CIFAR-100, we show that our method improves state-of-the-art detector robustness against adversarial examples. Furthermore, we validate the energy efficiency of our proposed adversarial detection methodology through an extensive energy analysis on various hardware scalable CMOS accelerator platforms. We also demonstrate the effects of quantization on our detector-appended networks.
Introduction
Deep Neural Networks (DNNs) are notoriously brittle to adversarial examples—small, calculated perturbations that can deceive a classifier while remaining invisible to humans. While various defenses have been proposed, they often incur a "robustness tax," significantly increasing computational overhead or slowing down inference. This research introduces a structured methodology to protect DNNs by strategically placing detector subnetworks at the most sensitive layers, optimizing for both robustness and energy efficiency.
Methodology
The core of this approach is the strategic placement of detectors based on a novel sensitivity metric and the use of hardware-aware quantization.
Adversarial Noise Sensitivity (ANS)
The authors propose Adversarial Noise Sensitivity (ANS) as a heuristic to profile network architectures. ANS measures the root-mean-square error of activations between clean and adversarial inputs at each layer. By identifying layers where this sensitivity peaks, the researchers can determine the optimal location for a detector.
As illustrated in the figure, sensitivity is not uniform across the network. Earlier layers often show higher ANS, allowing for earlier detection and potential computational savings.
Strategic Detector Placement and Early Exit
Instead of placing detectors at the end of a network, a simple binary classifier is appended to the layer with the highest ANS. If the detector identifies an input as adversarial, the network performs an early exit, halting computation immediately. This prevents the adversarial noise from propagating through the entire network and reduces total energy consumption.
Hardware-Optimized Quantization
To ensure practicality for edge devices, the main network is quantized to 12-bit precision, while the detector is reduced to a 1-bit (binary) precision. The efficiency of this approach is evaluated using CMOS accelerator simulations, specifically focusing on Data Gating (DG) and Dynamic Voltage-Accuracy and Frequency Scaling (DVAFS).
Experimental Results
The methodology was evaluated on MNIST, CIFAR-10, and CIFAR-100 benchmarks against FGSM and PGD attacks.
- Energy Efficiency: The detector-augmented ResNet18 and VGG19 models achieved energy reductions of 17% and 20%, respectively, compared to standard 16-bit baselines.
- Detection Robustness: In white-box attacks, the detectors achieved an AUC of 1.0. The system also demonstrated resilience against dynamic attackers through adversarial training of the detector.
- Quantization Success: Results showed that while the primary model requires at least 12 bits for accuracy, the detector remains highly effective at 1-bit precision, significantly lowering hardware implementation costs.
Conclusion
By leveraging the ANS metric, this work provides a structured way to integrate adversary detection into DNNs. The strategic placement of low-precision detectors enables early exits that enhance both security and energy efficiency. Future research will focus on scaling these methods to larger datasets like ImageNet and exploring multi-exit strategies for more granular control.
Summary updated