Model Agnostic Dual Quality Assessment for Adversarial Machine Learning and an Analysis of Current Neural Networks and Defenses

Danilo Vasconcellos Vargas, Shashank KotyanarXiv 2019 · arXiv

Introduction

Evaluating the robustness of neural networks against adversarial attacks is currently a fragmented and often biased process. Researchers have developed numerous methods to fool models (attacks) and protect them (defenses), but comparing these techniques is difficult due to model dependence, insufficient metrics (like over-reliance on $L_1$ and $L_2$), and the creation of "false" adversarial samples that are unrecognizable even to humans. This paper proposes a dual quality assessment framework that uses two complementary black-box attacks—$L_0$ and $L_\infty$—to provide a model-agnostic, human-verifiable measure of robustness.

Methodology

The authors propose a standardized testing framework using two specific types of black-box attacks that do not require knowledge of a model's internal architecture or gradients:

  • Few-Pixel ($L_0$) Attack: This variation of the "One-Pixel Attack" modifies a very small number of pixels (e.g., 1, 3, 5, or 10) to induce misclassification.
  • Threshold ($L_\infty$) Attack: A novel black-box attack that modifies every pixel in the image, but only by a tiny, predefined amount (a threshold).

To optimize these perturbations without gradients, the researchers employed evolutionary algorithms, specifically Differential Evolution (DE) and CMA-ES (Covariance Matrix Adaptation Evolution Strategy). They defined "Robustness Levels" (1, 3, 5, 10) based on the number of pixels or the intensity of the threshold, allowing for a fair comparison across different models. The study primarily utilized the CIFAR-10 dataset to evaluate seven state-of-the-art architectures, including WideResNet, ResNet, DenseNet, CapsNet, and LeNet.

Visual examples of Few-Pixel ($L_0$) and Threshold ($L_\infty$) attacks across various neural network architectures.

Key Insights and Analysis

A critical finding of this research is that robustness is not a monolithic trait; a model can be resistant to $L_0$ attacks while remaining highly vulnerable to $L_\infty$ attacks, and vice versa. For example, CapsNet (Capsule Networks) proved to be the most robust against few-pixel attacks but was highly vulnerable to threshold attacks.

The authors also provide a mathematical and visual argument for why traditional $L_1$ and $L_2$ metrics often fail. These metrics do not guarantee that noise is spread out, often allowing "concentrated" noise that makes an image unrecognizable. By contrast, $L_0$ and $L_\infty$ strictly constrain the spatial distribution of noise, ensuring that adversarial examples remain visually consistent with their original class for human observers.

Results and Quantitative Evaluation

The experiments revealed that no single model or defense mechanism is entirely safe; every tested network was vulnerable at some level of the dual assessment. Quantitatively, using the Area Under the Curve (AUC) for robustness plots, AllConv and CapsNet were generally the most robust, though their strengths lay in different categories of the dual framework.

Heatmaps illustrating the attack success rates for different model architectures and CIFAR-10 classes under the dual assessment framework.

Furthermore, the study identified significant class sensitivity. In CIFAR-10, "bird" and "cat" classes were consistently easier to attack than "frog" or "truck," likely because their features are more easily confused by the models. Common defenses like Feature Squeezing and Total Variance Minimization showed a lack of consistency, often failing against one of the two dual attack types.

Conclusion and Future Work

The dual quality assessment framework provides a more rigorous and human-centric way to evaluate model robustness. While effective on the CIFAR-10 dataset, the computational expense of evolutionary black-box attacks remains a hurdle for scaling to high-resolution datasets like ImageNet. Future work should focus on developing hybrid models that combine the architectural strengths necessary to achieve high robustness across both $L_0$ and $L_\infty$ metrics simultaneously.

Summary updated

Abstract

There exists a vast number of adversarial attacks and defences for machine learning algorithms of various types which makes assessing the robustness of algorithms a daunting task. To make matters worse, there is an intrinsic bias in these adversarial algorithms. Here, we organise the problems faced: a) Model Dependence, b) Insufficient Evaluation, c) False Adversarial Samples, and d) Perturbation Dependent Results). Based on this, we propose a model agnostic dual quality assessment method, together with the concept of robustness levels to tackle them. We validate the dual quality assessment on state-of-the-art neural networks (WideResNet, ResNet, AllConv, DenseNet, NIN, LeNet and CapsNet) as well as adversarial defences for image classification problem. We further show that current networks and defences are vulnerable at all levels of robustness. The proposed robustness assessment reveals that depending on the metric used (i.e., $L_0$ or $L_\infty$), the robustness may vary significantly. Hence, the duality should be taken into account for a correct evaluation. Moreover, a mathematical derivation, as well as a counter-example, suggest that $L_1$ and $L_2$ metrics alone are not sufficient to avoid spurious adversarial samples. Interestingly, the threshold attack of the proposed assessment is a novel $L_\infty$ black-box adversarial method which requires even less perturbation than the One-Pixel Attack (only $12\%$ of One-Pixel Attack's amount of perturbation) to achieve similar results. Code is available at http://bit.ly/DualQualityAssessment.
BibTeX
@misc{vargas2019model,
  author = {Danilo Vasconcellos Vargas and Shashank Kotyan},
  title = {{Model Agnostic Dual Quality Assessment for Adversarial Machine Learning and an Analysis of Current Neural Networks and Defenses}},
  year = {2019},
  eprint = {1906.06026},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.06026},
}
Figure