Interpretable BoW Networks for Adversarial Example Detection
Introduction
Deep learning models, particularly Convolutional Neural Networks (CNNs), have achieved remarkable success in computer vision. However, they remain notoriously opaque "black boxes." While a model might correctly identify a "dog," understanding the internal logic behind that classification is challenging. This lack of interpretability is not just a theoretical concern but a significant security vulnerability. Adversarial attacks—subtle, human-imperceptible perturbations—can easily trick these models into misclassifying critical objects, such as mistaking a "stop sign" for a "speed limit sign."
Existing interpretability methods often rely on post-hoc analyses, such as saliency maps, which highlight influential pixels but fail to reveal the network's internal "prototypes" for specific classes. This research addresses the need for intrinsic interpretability by developing a model that utilizes human-understandable concepts for decision-making and leverages this transparency to detect adversarial threats.
Methodology: The Interpretable BoW Architecture
The authors propose Interpretable BoW Networks, which integrate a Bag of visual Words (BoW) layer into the CNN architecture. This approach shifts the model toward a prototype-based classification scheme, more akin to human categorization.
The system consists of three primary components:
- The BoW Layer: Instead of standard global pooling or flattening, a BoW layer is used. It compares local image features to a "codebook" of learned codewords (prototypes). Classification is then based on the activation levels of these codewords.
- GAN-based Visualization: To ensure the codewords are human-interpretable, the researchers employ a pre-trained Generative Adversarial Network (GAN). By optimizing latent vectors to match specific codeword features, they generate images that represent the network's internal "visual dictionary."
- Adversarial Detection: The core defensive mechanism relies on identifying semantic mismatches. In an adversarial attack (e.g., a "dog" image modified to be classified as a "cat"), the network will activate a "cat" codeword. However, the visual representation of that codeword will not match the input image's actual appearance. A two-stream detector compares the input to the visual prototype of the most active codeword; a significant discrepancy flags the input as an adversarial or out-of-distribution (OOD) sample.
Visualizing the Dictionary
The model's interpretability was evaluated across four standard benchmarks: MNIST (digits), Fashion-MNIST (clothing), SVHN (house numbers), and CIFAR-10 (general objects). By visualizing the learned codewords, researchers can inspect the "prototypes" the network uses for each class.
As shown in the visual dictionary, the codewords capture the essential semantic features of the target classes. For instance, in MNIST, codewords clearly represent digit shapes, while in CIFAR-10, they capture distinct object parts or textures associated with classes like "aeroplane" or "bird." This transparency allows users to verify if the model is learning meaningful features rather than exploiting dataset biases.
Key Results and Insights
The Interpretable BoW Network demonstrates that transparency does not have to come at the cost of performance.
- Classification Accuracy: The BoW-based models achieved performance nearly identical to standard CNNs. For example, on MNIST, the BoW network reached 99.15% accuracy compared to 99.23% for a standard CNN.
- Superior Detection: The proposed semantic mismatch detection consistently outperformed state-of-the-art baselines. On the CIFAR-10 dataset, the detector achieved an AUROC of 99.96% against BIM-b adversarial attacks, significantly higher than the 82.79% achieved by Local Intrinsic Dimensionality (LID).
- Robustness and OOD Detection: The BoW architecture itself proved more inherently robust to white-box attacks. Furthermore, the system excelled at identifying OOD samples—images from datasets outside the training distribution—often achieving 100% detection rates.
Conclusion and Future Directions
While highly effective, the Interpretable BoW framework involves a multi-stage training process (GAN, base network, and BoW layer) that is more computationally intensive than standard training. Additionally, the quality of interpretability is currently dependent on the GAN's generative capabilities. Future work may explore "part-based" BoW representations to better handle complex scenes by breaking objects down into constituent components. Despite these challenges, this work represents a significant step toward building deep learning systems that are both high-performing and fundamentally trustworthy.
Summary updated