Detection of Uncertainty in Exceedance of Threshold (DUET): An Adversarial Patch Localizer

Terence Jie Chua, Wenhan Yu, Chang Liu, Jun Zhao

TL;DR

The DUET framework introduces a Bayesian approach to not just detect, but precisely localize adversarial patches in images while providing a "safety buffer" through uncertainty quantification. By replacing standard neural network layers with Bayesian ones, the authors create a system that can flag suspicious image regions with high confidence, enabling safety-critical applications like autonomous driving to ignore or reconstruct compromised data.

What Problem Does This Paper Address?

Deep learning models are notoriously vulnerable to adversarial patches—small, localized perturbations (like a sticker on a stop sign) that can completely derail a model's prediction.

While many defenses exist, they often fall into two traps:

  1. Performance Degradation: They often make the model worse at processing normal, "clean" data.
  2. Lack of Spatial Awareness: Most detectors only say "this image is attacked" without telling the system where the attack is happening.

In safety-critical fields like medical imaging or self-driving cars, simply knowing an attack exists isn't enough. You need to know which pixels to distrust and exactly how much confidence you should have in that distrust.

Methods

The authors propose DUET, a framework built on the DeepLabV3+ segmentation architecture. The "secret sauce" is the integration of Bayesian Neural Networks (BNNs).

  • Bayesian Backbone: Instead of fixed weights, the model learns a probability distribution over weights using Variational Inference. This allows the model to perform Monte Carlo (MC) sampling during inference to produce a distribution of possible outputs for every pixel.
  • Uncertainty Calibration: BNNs often underestimate their own uncertainty. The authors use an extension of Platts Scaling and Isotonic Regression to calibrate the output, ensuring that a "90% confidence interval" actually contains the true value 90% of the time.
  • The DUET Algorithm: The core logic is "Exceedance of Threshold." For a pixel to be deemed "safe" (non-adversarial), the entire upper bound of its calibrated confidence interval must fall below a specific classification threshold. If there's any statistical doubt, the pixel is flagged as adversarial.

Datasets

The model was trained and evaluated on three diverse datasets to test its ability to generalize:

  • Flower102: A dataset of flower species.
  • GTSRB (German Traffic Sign Recognition Benchmark): Crucial for testing "real-world" physical attack scenarios.
  • Food101: Used primarily to test performance on clean, non-adversarial images.

The authors tested against several attack types, including PGD (Projected Gradient Descent), FGSM (Fast Gradient Sign Method), and the Google Patch attack.

Key Insights and Ideas

  • Layer Placement Matters: One of the most interesting findings is that where you put the Bayesian layers changes the model's behavior. Adding Bayesian layers to the earlier blocks of the encoder (closer to the input) leads to better top-end localization performance. Adding them to later blocks improves the model's stability and ability to generalize to clean data.
  • Priors Aren't One-Size-Fits-All: While complex priors like Cauchy or Gaussian Mixture Models (GMM) are popular in research, the authors found that simple Isometric Gaussian priors actually performed best for the specific task of patch localization.
  • The "Safety First" Heuristic: By using the 95th percentile (one-tail) of the uncertainty distribution, the model acts conservatively. It would rather "over-flag" a suspicious pixel than let a potentially adversarial one pass as benign.

Key Results

  • Superior Localization: On the Flower102-PGD test set, the DUET model achieved a mean Intersection over Union (mIoU) of 0.921, significantly outperforming the non-Bayesian baseline of 0.868.
  • Robustness to Unseen Attacks: When faced with the Google Patch attack (which it wasn't trained on), the DUET variants maintained high performance (mIoU ~0.85-0.90), while the baseline model's performance was much more erratic.
  • Clean Data Integrity: On the Food101 "clean" dataset, the best DUET variants achieved near-perfect segmentation (mIoU of 0.999), whereas the non-Bayesian baseline struggled with a much lower mIoU of 0.854, often confusing natural image features for adversarial patches.

Limitations and Future Work

  • Inference Latency: Because BNNs require multiple forward passes (MC sampling) to generate uncertainty estimates, they are computationally more expensive than standard models. This could be a hurdle for real-time applications.
  • The "Perfect" Balance: The authors noted they couldn't find a single configuration that was simultaneously the best at localization and the best at clean data performance; there is still a slight trade-off between the two.
  • Decoder Exploration: This study focused heavily on the encoder. Future work could investigate how adding Bayesian layers to the decoder section of the segmentation network might further refine patch boundaries.

Summary updated

Abstract

Development of defenses against physical world attacks such as adversarial patches is gaining traction within the research community. We contribute to the field of adversarial patch detection by introducing an uncertainty-based adversarial patch localizer which localizes adversarial patch on an image, permitting post-processing patch-avoidance or patch-reconstruction. We quantify our prediction uncertainties with the development of Detection of Uncertainties in the Exceedance of Threshold (DUET) algorithm. This algorithm provides a framework to ascertain confidence in the adversarial patch localization, which is essential for safety-sensitive applications such as self-driving cars and medical imaging. We conducted experiments on localizing adversarial patches and found our proposed DUET model outperforms baseline models. We then conduct further analyses on our choice of model priors and the adoption of Bayesian Neural Networks in different layers within our model architecture. We found that isometric gaussian priors in Bayesian Neural Networks are suitable for patch localization tasks and the presence of Bayesian layers in the earlier neural network blocks facilitates top-end localization performance, while Bayesian layers added in the later neural network blocks contribute to better model generalization. We then propose two different well-performing models to tackle different use cases.
BibTeX
@inproceedings{chua2022detection,
  author = {Terence Jie Chua and Wenhan Yu and Chang Liu and Jun Zhao},
  title = {{Detection of Uncertainty in Exceedance of Threshold (DUET): An Adversarial Patch Localizer}},
  booktitle = {BDCAT},
  year = {2022},
  url = {https://openreview.net/forum?id=DxvjY1HuNp},
}

Loading related content...

Figure