Defending Black Box Facial Recognition Classifiers Against Adversarial Attacks

Rajkumar Theagarajan, Bir BhanuWorkshops @ CVPR 2020 · DOI

Abstract

Adversarial attacks pose a significant threat to facial recognition systems, particularly in "Black Box" scenarios where the internal architecture of the classifier is unknown. This paper presents a novel defense framework that utilizes a substitute model and Bayesian Neural Networks (BNNs) to estimate uncertainty as a proxy for adversarial presence. By implementing an iterative loop involving an ensemble of image purifiers and a rejection mechanism for high-uncertainty samples, the system effectively mitigates the impact of adversarial perturbations without requiring access to the target model's internals. Experimental results on the MS-Celeb dataset demonstrate substantial improvements in classification accuracy and robust rejection of highly corrupted samples.

Introduction

Adversarial attacks are the "optical illusions" of the AI world: tiny, human-imperceptible perturbations added to an image that cause a deep learning model to confidently misclassify it. Most existing defenses assume a "White Box" scenario where the model architecture is known. However, real-world biometrics—such as airport security or remote monitoring—often rely on proprietary "Black Box" classifiers.

In live systems, the lack of ground truth labels makes it difficult to validate if a defense has successfully purified an image. The authors address a critical challenge: How can we validate if an image is "adversarial-free" without knowing the model's internals or the true identity of the subject? This work proposes using model uncertainty as a reliable metric for detecting and neutralizing these attacks.

Proposed Defense Framework

The authors propose a multi-stage defense pipeline that functions independently of the original Black Box classifier. The architecture is centered around a substitute model and an iterative purification process.

1. Knowledge Distillation and the Substitute Model

Since the Black Box is inaccessible, the authors "probe" it with thousands of images to observe its outputs. These reactions are used to train a Substitute Model that mimics the Black Box's behavior.

2. Bayesian CNN Detector

The substitute model is converted into a Bayesian Neural Network. Unlike standard models that provide a single point estimate, a Bayesian model provides a measure of uncertainty. It specifically tracks Epistemic uncertainty, which arises from the model's lack of knowledge about a specific input—a key indicator of adversarial manipulation.

3. Iterative Purification Loop

If the Bayesian CNN detects high uncertainty, the image is sent to an ensemble of purifiers (PixelDefend and MagNet). These tools attempt to strip away adversarial noise. The purified image is then fed back into the Bayesian CNN. If the uncertainty remains above a defined threshold, it undergoes another round of purification, repeating up to $M$ times.

Figure 1: Flowchart of the proposed iterative defense system using Bayesian CNN and an ensemble of purifiers.

4. Rejection Mechanism

For safety-critical biometrics, rejection is a feature. If an image still shows high uncertainty after $M$ iterations, the system rejects it entirely. This prevents the system from making dangerous guesses on heavily corrupted images that might be discernible to humans but confusing to AI.

Experimental Setup

The study utilizes the MS-Celeb dataset, containing roughly 9.5 million images. To ensure data quality, the authors applied a graph-based cleaning method to remove mislabeled images. For testing, 100 celebrities with at least 100 images each were selected to build the evaluation cases.

Results and Discussion

The framework was tested against three major attack types: FGSM, BIM, and PGD at various noise levels ($\epsilon$).

Uncertainty as a Proxy for Accuracy

The paper demonstrates that high uncertainty correlates strongly with adversarial presence. In a live system where accuracy cannot be calculated in real-time, uncertainty serves as a vital proxy for trust.

Effectiveness of Iterative Purification

The iterative approach proves essential for reducing adversarial noise. As shown in the experimental data, the "amount of purification" required decreases over successive iterations as the image is restored to a cleaner state.

Figure 2: Graph showing the amount of purification required over multiple iterations for different attack types and noise levels.

Performance Metrics

  • Superior Accuracy: At a noise level of $\epsilon = 0.05$, the "No Defense" scenario resulted in ~25-30% accuracy. The proposed ensemble boosted this to ~68-70%.
  • Handling High Perturbation: When noise was increased to $\epsilon = 0.2$ (visible distortion), the Bayesian CNN's uncertainty metrics spiked, allowing the system to successfully reject 100% of those samples.

Conclusion

This research provides a robust method for defending proprietary facial recognition systems. By leveraging Bayesian uncertainty and iterative purification, the framework significantly improves security against black-box attacks. Future work will focus on reducing the computational overhead of Bayesian sampling and testing against physical adversarial attacks, such as adversarial glasses or masks.

Summary updated

BibTeX
@inproceedings{theagarajan2020defending,
  author = {Rajkumar Theagarajan and Bir Bhanu},
  title = {{Defending Black Box Facial Recognition Classifiers Against Adversarial Attacks}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00414},
}
Figure