Adversarially Robust Classification Based on GLRT

Bhagyashree Puranik, Upamanyu Madhow, Ramtin PedarsaniICASSP 2021 · DOI

TL;DR

This paper proposes a new defense strategy for machine learning models against adversarial attacks, called the Generalized Likelihood Ratio Test (GLRT). The GLRT approach offers a competitive robustness-accuracy trade-off compared to existing minimax strategies, especially against weaker attacks, and generalizes naturally to more complex models.

What Problem Does This Paper Address?

Machine learning models, particularly deep neural networks, are highly susceptible to adversarial attacks. These attacks involve adding small, carefully crafted perturbations to input data, which can cause misclassification with high confidence. While many defense mechanisms have been proposed, they often rely on empirical training with adversarial examples, lacking theoretical guarantees or deep insights into their effectiveness. This paper aims to provide a more fundamental understanding of adversarial robustness by framing classification as a hypothesis testing problem and proposing a defense based on the Generalized Likelihood Ratio Test (GLRT).

Methods

The core of the paper's contribution is the application of the Generalized Likelihood Ratio Test (GLRT) as a defense mechanism against adversarial attacks. In the context of hypothesis testing, the GLRT jointly estimates both the class of interest and the parameters of the adversarial perturbation, treating the perturbation as a nuisance parameter. This approach is general and can be applied to composite hypothesis testing problems.

The authors specifically analyze the GLRT for binary Gaussian hypothesis testing with $\ell_\infty$-bounded adversarial perturbations. In this setting, they compare the GLRT to the optimal minimax strategy, for which a solution is known. The GLRT detector is derived by minimizing a cost function that involves a coordinate-wise application of a double-sided ReLU function to the difference between the observation and the signal templates. This is a modification of the standard minimum distance rule.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • GLRT as a principled defense: The paper demonstrates that GLRT, a well-established statistical tool, can be effectively repurposed as a defense against adversarial attacks, offering a more theoretically grounded approach than purely empirical methods.
  • Robustness-Accuracy Trade-off: GLRT provides a better robustness-accuracy trade-off than minimax strategies for weaker attacks, suggesting it's more adaptable to varying attack strengths.
  • Generalizability: The GLRT defense is shown to generalize naturally to more complex models where optimal minimax classifiers are difficult or impossible to derive.
  • Insight into Adversarial Robustness: By analyzing the GLRT in a simplified setting, the paper provides insights into why certain features (like signal sparsity) contribute to robustness.
  • GLRT vs. Minimax: While minimax strategies are optimal against the worst-case attack, GLRT can be more practical and efficient when the actual attack is weaker than the worst-case scenario.

Key Results

  • For binary Gaussian hypothesis testing with $\ell_\infty$-bounded perturbations, the GLRT approach yields performance competitive with the minimax approach under the worst-case attack.
  • GLRT offers a better robustness-accuracy trade-off under weaker attacks compared to minimax strategies.
  • A Central Limit Theorem (CLT) based approximation is used to analyze the performance of the GLRT detector, showing good agreement with simulations for moderately large dimensions.
  • The paper presents numerical examples and plots comparing the error probabilities of GLRT and minimax detectors under varying attack budgets, illustrating GLRT's advantage for weaker attacks.

Limitations and Future Work

  • Analytical Complexity: The paper acknowledges that the nonlinear nature of the GLRT rule makes its performance characterization more difficult than that of linear detectors.
  • Generalization to Complex Models: While the paper states GLRT generalizes, further research is needed to explore its application and effectiveness on more complex data and attack models (e.g., deep neural networks).
  • Minimax Formulation for Complex Models: The authors suggest that while finding optimal minimax rules for complex models is challenging, insights from both minimax and GLRT formulations for simpler models could be combined for better understanding.
  • Further Research Directions: The paper explicitly mentions exploring GLRT for more complex data and attack models and investigating the minimax formulation in such settings.

Summary updated

BibTeX
@inproceedings{puranik2021adversarially,
  author = {Bhagyashree Puranik and Upamanyu Madhow and Ramtin Pedarsani},
  title = {{Adversarially Robust Classification Based on GLRT}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  doi = {10.1109/icassp39728.2021.9413587},
}
Figure