Detection of Uncertainty in Exceedance of Threshold (DUET): An Adversarial Patch Localizer

Terence Jie Chua, Wenhan Yu, Jun Zhao

Abstract

Development of defenses against physical world attacks such as adversarial patches is gaining traction within the research community. We contribute to the field of adversarial patch detection by introducing an uncertainty-based adversarial patch localizer which localizes adversarial patch on an image, permitting post-processing patch-avoidance or patch-reconstruction. We quantify our prediction uncertainties with the development of Detection of Uncertainties in the Exceedance of Threshold (DUET) algorithm. This algorithm provides a framework to ascertain confidence in the adversarial patch localization, which is essential for safety-sensitive applications such as self-driving cars and medical imaging. We conducted experiments on localizing adversarial patches and found our proposed DUET model outperforms baseline models. We then conduct further analyses on our choice of model priors and the adoption of Bayesian Neural Networks in different layers within our model architecture. We found that isometric gaussian priors in Bayesian Neural Networks are suitable for patch localization tasks and the presence of Bayesian layers in the earlier neural network blocks facilitates top-end localization performance, while Bayesian layers added in the later neural network blocks contribute to better model generalization. We then propose two different well-performing models to tackle different use cases.

Introduction

Deep learning models are notoriously vulnerable to adversarial patches—small, localized perturbations, such as a sticker on a traffic sign, that can completely derail a model's prediction. While many defenses exist, they often suffer from two primary drawbacks: performance degradation on clean data and a lack of spatial awareness. Most detectors merely flag an image as attacked without identifying the specific compromised region. In safety-critical fields like medical imaging or autonomous driving, simply knowing an attack exists is insufficient; systems must identify which pixels to distrust and quantify the confidence in that assessment.

Methodology: The DUET Framework

The proposed DUET framework is built upon the DeepLabV3+ segmentation architecture, integrated with Bayesian Neural Networks (BNNs). Unlike standard neural networks with fixed weights, BNNs learn a probability distribution over weights using Variational Inference. This allows the model to perform Monte Carlo (MC) sampling during inference to produce a distribution of possible outputs for every pixel.

The DUET architecture integrates a Bayesian encoder (ResNet-101) with a calibrator and thresholding logic to produce adversarial patch predictions.

Uncertainty Calibration and the DUET Algorithm

Because BNNs often underestimate their own uncertainty, the authors employ an extension of Platts Scaling and Isotonic Regression to calibrate the output. The core logic of the framework is the "Exceedance of Threshold" algorithm. For a pixel to be deemed "safe" (non-adversarial), the entire upper bound of its calibrated confidence interval (e.g., the 95th percentile) must fall below a specific classification threshold. If there is statistical doubt, the pixel is flagged as possibly adversarial.

The DUET logic uses uncertainty distributions from N samples to distinguish between non-adversarial and possibly adversarial pixels based on whether the confidence interval exceeds a threshold.

Experimental Results

The model was evaluated on the Flower102, GTSRB (German Traffic Sign Recognition Benchmark), and Food101 datasets against various attack types, including PGD, FGSM, and the Google Patch attack.

  • Superior Localization: On the Flower102-PGD test set, the DUET model achieved a mean Intersection over Union (mIoU) of 0.921, significantly outperforming the non-Bayesian baseline of 0.868.
  • Robustness to Unseen Attacks: When faced with the Google Patch attack (unseen during training), DUET variants maintained high performance (mIoU ~0.85-0.90), while the baseline model's performance was erratic.
  • Clean Data Integrity: On the Food101 "clean" dataset, the best DUET variants achieved near-perfect segmentation (mIoU of 0.999), whereas the baseline struggled (mIoU 0.854), often confusing natural image features for adversarial patches.

Discussion and Insights

Research into model architecture revealed that layer placement is critical: adding Bayesian layers to earlier encoder blocks improves top-end localization, while adding them to later blocks enhances model stability and generalization. Furthermore, simple Isometric Gaussian priors were found to be more effective for patch localization than complex alternatives like Cauchy or Gaussian Mixture Models.

While DUET provides a robust safety buffer, the computational cost of MC sampling remains a hurdle for real-time applications. Future work will investigate the integration of Bayesian layers into the decoder section to further refine patch boundaries.

Summary updated

Abstract

Development of defenses against physical world attacks such as adversarial patches is gaining traction within the research community. We contribute to the field of adversarial patch detection by introducing an uncertainty-based adversarial patch localizer which localizes adversarial patch on an image, permitting post-processing patch-avoidance or patch-reconstruction. We quantify our prediction uncertainties with the development of \textit{\textbf{D}etection of \textbf{U}ncertainties in the \textbf{E}xceedance of \textbf{T}hreshold} (DUET) algorithm. This algorithm provides a framework to ascertain confidence in the adversarial patch localization, which is essential for safety-sensitive applications such as self-driving cars and medical imaging. We conducted experiments on localizing adversarial patches and found our proposed DUET model outperforms baseline models. We then conduct further analyses on our choice of model priors and the adoption of Bayesian Neural Networks in different layers within our model architecture. We found that isometric gaussian priors in Bayesian Neural Networks are suitable for patch localization tasks and the presence of Bayesian layers in the earlier neural network blocks facilitates top-end localization performance, while Bayesian layers added in the later neural network blocks contribute to better model generalization. We then propose two different well-performing models to tackle different use cases.
BibTeX
@inproceedings{chua2022detection,
  author = {Terence Jie Chua and Wenhan Yu and Jun Zhao},
  title = {{Detection of Uncertainty in Exceedance of Threshold (DUET): An Adversarial Patch Localizer}},
  booktitle = {BDCAT},
  year = {2022},
  url = {https://openreview.net/forum?id=DxvjY1HuNp},
}
Figure