Adversarial Discriminative Attention for Robust Anomaly Detection

Daiki Kimura, Subhajit Chaudhury, Minori Narita, Asim Munawar, Ryuki TachibanaWACV 2020 · OpenReview

Abstract

Existing methods for visual anomaly detection predominantly rely on global level pixel comparisons for anomaly score computation without emphasizing on unique local features. However, images from real-world applications are susceptible to unwanted noise and distractions, that might jeopardize the robustness of such anomaly score. To alleviate this problem, we propose a self-supervised masking method that specifically focuses on discriminative parts of images to enable robust anomaly detection. Our experiments reveal that discriminator's class activation map in adversarial training evolves in three stages and finally fixates on the foreground location in the images. Using this property of the activation map, we construct a mask that suppresses spurious signals from the background thus enabling robust anomaly detection by focusing on local discriminative attributes. Additionally, our method can further improve the accuracy by learning a semi-supervised discriminative classifier in cases where a few samples from anomaly classes are available during the training. Experimental evaluations on four different types of datasets demonstrate that our method outperforms previous state-of-the-art methods for each condition and in all domains.

Introduction

Visual anomaly detection systems, typically built on Variational Autoencoders (VAEs) or Generative Adversarial Networks (GANs), often struggle with real-world noise. These models generally rely on global pixel-level reconstruction errors: they learn a representation of "normal" data and flag any significant deviation as an anomaly. However, in practical settings—such as industrial inspection or satellite monitoring—background noise, lighting changes, or shadows can cause high reconstruction errors in normal samples, leading to false positives. Humans naturally ignore these distractions by focusing on salient objects. This paper proposes a method to bridge this gap by teaching models to focus on discriminative features while suppressing background noise.

Methodology: Adversarial Discriminative Attention

The core of the proposed approach is a VAEGAN architecture enhanced with a self-supervised masking mechanism. The system leverages the discriminator's internal "attention" to identify salient regions of the image.

The proposed architecture uses a discriminator's class activation maps to mask reconstruction errors, focusing the anomaly score on salient regions.

The process involves three key components:

  1. Self-Supervised Masking: Using Grad-CAM (Gradient-weighted Class Activation Mapping) on the discriminator network, the model generates a heatmap indicating which regions are most relevant for distinguishing real from fake data.
  2. Attention-Weighted Scoring: This heatmap acts as a mask. The final anomaly score is calculated by multiplying the raw reconstruction error by this mask, effectively "muting" errors originating from non-discriminative background areas.
  3. Semi-Supervised Extension: When a small set of known anomalies is available, a second discriminator can be trained to further refine the attention mask, focusing specifically on features that characterize known defects.

Experimental Results

The method was evaluated across four datasets: Noisy MNIST, Hand Gestures, Pigeon Gestures (for medical screening), and Venus satellite imagery. In all cases, the attention-masked approach demonstrated superior robustness to noise compared to standard baselines like AnoGAN and AnoVAEGAN.

Qualitative comparison showing how the proposed attention mask (Ours) correctly identifies anomalies while ignoring background noise that confuses traditional methods.

On the Noisy MNIST dataset, the proposed method achieved an AUROC of 0.74 for digit "0" detection, outperforming AnoVAEGAN (0.71). In the Hand Gesture dataset, the semi-supervised variant reached an AUROC of 0.948. Qualitative results illustrate that while previous methods were often "tricked" by background noise—assigning high anomaly scores to normal samples—the attention-masked version correctly suppressed these spurious signals.

Discussion: Discriminator Evolution

A key insight from this work is the observation of how the discriminator's attention evolves during training. It typically progresses through three stages: early broad attention, a middle stage focusing on background artifacts, and a late stage fixating on the foreground salient object. This natural evolution justifies the use of the final-stage CAM as a reliable mask for identifying the object of interest without requiring manual labels or bounding boxes.

Conclusion

By integrating self-supervised attention masks into the anomaly detection pipeline, this research provides a robust solution for real-world visual inspection tasks. While the current model is optimized for foreground anomalies, future work may explore temporal consistency in video data or joint learning of activation maps to further enhance detection capabilities.

Summary updated

Abstract

Existing methods for visual anomaly detection predominantly rely on global level pixel comparisons for anomaly score computation without emphasizing on unique local features. However, images from real-world applications are susceptible to unwanted noise and distractions, that might jeopardize the robustness of such anomaly score. To alleviate this problem, we propose a self-supervised masking method that specifically focuses on discriminative parts of images to enable robust anomaly detection. Our experiments reveal that discriminator's class activation map in adversarial training evolves in three stages and finally fixates on the foreground location in the images. Using this property of the activation map, we construct a mask that suppresses spurious signals from the background thus enabling robust anomaly detection by focusing on local discriminative attributes. Additionally, our method can further improve the accuracy by learning a semi-supervised discriminative classifier in cases where a few samples from anomaly classes are available during the training. Experimental evaluations on four different types of datasets demonstrate that our method outperforms previous state-of-the-art methods for each condition and in all domains.
BibTeX
@inproceedings{kimura2020adversarial,
  author = {Daiki Kimura and Subhajit Chaudhury and Minori Narita and Asim Munawar and Ryuki Tachibana},
  title = {{Adversarial Discriminative Attention for Robust Anomaly Detection}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=sNyUzUGXDM},
}
Figure