Abstract
Deep neural networks are susceptible to adversarial examples—inputs specifically crafted to induce misclassification through human-imperceptible perturbations. This paper investigates a defense strategy that expands the training set with a single, diverse "background class" intended to fill the empty spaces between classification boundaries. While this approach effectively detects and mitigates simple gradient-based attacks on the EMNIST dataset, it remains vulnerable to sophisticated, optimization-based attacks like Carlini-Wagner. We discuss the implications of these results and the limitations of using static background datasets for robust manifold filling.
Introduction: The Vulnerability of Closed-Set Classifiers
Deep neural networks are notoriously brittle. By applying tiny perturbations to an image, an attacker can trick a model into misclassifying a "4" as a "7." This vulnerability stems largely from the closed-set nature of standard classifiers. In a typical MNIST digit classifier, every point in the high-dimensional feature space must be assigned to one of the ten digit classes. There is no inherent concept of "none of the above."
Adversarial attacks exploit this by finding regions of the feature space that the model has not encountered during training but is forced to assign to a specific class. The goal of this research is to "fill in" these empty spaces to create a more robust decision boundary.
Methodology
The proposed defense expands the training set from $m$ classes to $m+1$ classes by introducing a "background" category. This category is intended to capture images that are distinct from the primary classes but exist within the same general domain.
The Background Class
The researchers utilized the EMNIST dataset, which includes both handwritten digits and letters. The primary classes consisted of digits 0–9. For the background class, the authors selected a subset of 22 distinct EMNIST letters. To ensure the model did not become confused, letters visually similar to digits (e.g., 'l' vs '1' or 'O' vs '0') were filtered out. The final training set included approximately 310,000 digits and 191,000 background letters.
Architecture and Evaluation
The model utilized a standard convolutional neural network (CNN) architecture featuring Batch Normalization and MaxOut layers, optimized via AdaDelta. The defense was tested against three attack vectors:
- Fast Gradient Sign Method (FGSM): A weak, one-step gradient attack.
- Fast Gradient (FG): A refined version of FGSM.
- Carlini-Wagner (CW): A strong, iterative optimization-based attack designed to find the minimal perturbation required for a specific misclassification.
Key Insights: Filling the Manifold
The core hypothesis is that by adding a background class, the decision boundaries for the target digits "shrink" to wrap more tightly around the actual training data. The "no man's land" between classes is no longer a sharp, arbitrary line where one digit immediately becomes another; instead, it is labeled as background.
This approach shifts the goal of defense from correction to detection. If a perturbed image is classified as "background," it serves as a tripwire, flagging the input as a likely adversarial attack rather than a legitimate digit.
Results
The defense demonstrated a clear trade-off between standard performance and robustness against simple attacks:
- Clean Accuracy: The addition of the background class resulted in only a negligible drop in accuracy on clean digits, moving from 99.6% to 99.1%.
- Weak Attack Mitigation: The defense significantly hampered simple attacks. Against FGSM, the success rate dropped from 92% to 68%. Against Fast Gradient, success dropped from 94% to 56%, with 43% of adversarial examples being correctly caught and labeled as "background."
- Strong Attack Failure: The defense provided 0% improvement against the Carlini-Wagner attack. Because CW is an optimization-based search, it simply treats the background class as another obstacle to navigate around, finding a path through the feature space that avoids the background buffer entirely.
Limitations and Future Work
Selecting an effective background class remains a significant challenge. If background images are too diverse, model capacity may be exhausted; if they are too similar to target classes, clean accuracy suffers.
Future research may move away from static datasets in favor of synthetic backgrounds. By creating "morphed" images—interpolating between two primary classes—researchers could specifically target and fill the most vulnerable gaps in the decision boundary. Additionally, larger network architectures may be required to maintain precision when handling the increased complexity of a multi-domain background class.
Summary updated