Exploring the role of Input and Output Layers of a Deep Neural Network in Adversarial Defense

Jay N. Paranjape, Rahul Kumar Dubey, Vijendran V. GopalanarXiv 2020 · OpenReview

Abstract

Deep neural networks are learning models having achieved state of the art performance in many fields like prediction, computer vision, language processing and so on. However, it has been shown that certain inputs exist which would not trick a human normally, but may mislead the model completely. These inputs are known as adversarial inputs. These inputs pose a high security threat when such models are used in real world applications. In this work, we have analyzed the resistance of three different classes of fully connected dense networks against the rarely tested non-gradient based adversarial attacks. These classes are created by manipulating the input and output layers. We have proven empirically that owing to certain characteristics of the network, they provide a high robustness against these attacks, and can be used in fine tuning other models to increase defense against adversarial attacks.

Introduction

Adversarial attacks—small, often imperceptible perturbations that trick models into misclassification—pose a major security risk for real-world AI applications. While many defenses exist for gradient-based attacks (which rely on knowing the model's internal gradients), they often fail against decision-based attacks.

Decision-based attacks, like the Boundary Attack, are "black-box": they require no information about the model's architecture or gradients, only the final output label. Because they are iterative and rely on the model's decision boundaries, they are exceptionally hard to defend against. This research explores how data manipulation at the entry (input) and exit (output) points of a network can disrupt these attacks.

Methodology

The study proposes and compares three modified model architectures against standard preprocessing techniques:

  1. IBOL (Inputs Binarized Outputs Logits): Inputs are binarized (pixels set to 0 or 1 based on a threshold of 127) before being passed to a standard classification head.
  2. INOI (Inputs Non-Binarized Outputs Images): The model performs Image Recall. Instead of outputting a class probability vector (logits), the network is trained to reconstruct a "representative image" of the target class. Classification is determined by finding the representative image with the lowest $L_2$ distance to the model's output.
  3. IBOI (Inputs Binarized Outputs Images): A combination of both—binarized inputs and an image recall output layer.
Comparison of standard classification (top path) and the proposed image recall task (bottom path).

Key Insights

Binarization as a Filter

By forcing pixels to be either 0 or 1, the model effectively ignores "slight" perturbations. An attacker must change a pixel significantly to cross the threshold, which often makes the adversarial noise visible to the human eye or changes the image so much that it's no longer a "subtle" attack.

Visualization of an input image (left), its noisy adversarial counterpart (center), and the effect of binarization/defense (right).

The Complexity of Recall

Traditional classification collapses high-dimensional input into a tiny output space (e.g., 10 numbers for 10 classes). Image recall keeps the output space high-dimensional. This makes it much harder for iterative attacks like the Boundary Attack to find a path toward a misclassification that still looks like the original image.

Results

The primary evaluation was conducted using the MNIST dataset. The results show that the proposed modifications, especially IBOI, vastly outperform traditional preprocessing:

Model Single Pixel Attack (SPA) Boundary Attack (BA)
CDR (Baseline) 0.10 0.04
TVM (Baseline) 0.15 0.05
IBOL 0.98 0.933
IBOI 0.966 1.0

The IBOI model achieved a 1.0 resistance score against the Boundary Attack, meaning the attack failed to find a successful adversarial example for any of the tested images.

Limitations and Future Work

While effective on MNIST, the "Image Recall" method struggled to scale to more complex datasets like CIFAR-10. The authors could not achieve even 20% accuracy on CIFAR-10 using a ResNet-based recall model, as intra-class variance is much higher than in handwritten digits. Future work will focus on more sophisticated "representative image" generation and advanced binarization ensembles to handle natural images.

Summary updated

Abstract

Deep neural networks are learning models having achieved state of the art performance in many fields like prediction, computer vision, language processing and so on. However, it has been shown that certain inputs exist which would not trick a human normally, but may mislead the model completely. These inputs are known as adversarial inputs. These inputs pose a high security threat when such models are used in real world applications. In this work, we have analyzed the resistance of three different classes of fully connected dense networks against the rarely tested non-gradient based adversarial attacks. These classes are created by manipulating the input and output layers. We have proven empirically that owing to certain characteristics of the network, they provide a high robustness against these attacks, and can be used in fine tuning other models to increase defense against adversarial attacks.
BibTeX
@misc{paranjape2020exploring,
  author = {Jay N. Paranjape and Rahul Kumar Dubey and Vijendran V. Gopalan},
  title = {{Exploring the role of Input and Output Layers of a Deep Neural Network in Adversarial Defense}},
  year = {2020},
  eprint = {2006.01408},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.01408},
}
Figure