Red PANDA: Disambiguating Anomaly Detection by Removing Nuisance Factors

Niv Cohen, Jonathan Kahana, Yedid HoshenarXiv 2022 · DOI · arXiv

Abstract

Anomaly detection (AD) typically identifies any deviation from a training distribution as an outlier. However, many variations—such as lighting, pose, or style—are semantically irrelevant "nuisance factors" that lead to false positives. We present Red PANDA (REpresentation Disentanglement for Pretrained Anomaly Detection Adaptation), a framework that utilizes domain-supervised disentanglement to filter out these specified nuisances. By learning to ignore irrelevant attributes, Red PANDA significantly improves the precision of semantic anomaly detection and mitigates algorithmic bias.

Introduction

In machine learning, the definition of an "anomaly" is often subjective. A standard detector trained on a dataset where specific attributes are correlated (e.g., all taxi drivers are female) might flag a male taxi driver as an anomaly. While statistically an outlier, such a case is semantically normal. This is the nuisance factor problem. Current AD methods struggle to distinguish between true semantic anomalies (e.g., a car driving the wrong way) and nuisance variations (e.g., a change in camera angle or style).

The authors propose Negative Attribute Guided Anomaly Detection (NAGAD), a paradigm where users specify attributes the model should ignore. This approach acknowledges that while defining every possible anomaly is impossible, identifying irrelevant variations is often straightforward for domain experts.

Methods

Red PANDA adapts pretrained representations through a three-stage process:

  1. Contrastive Disentanglement: Using an encoder initialized with ImageNet weights, the model applies a specialized contrastive loss to group training samples by nuisance labels. It trains the encoder to produce representations invariant to these labels, effectively "un-seeing" the nuisance attributes.
  2. Conditional Reconstruction: To prevent the loss of relevant semantic information, a conditional generator must reconstruct the original image using the "clean" representation combined with the nuisance label. This ensures the representation remains rich in semantic content.
  3. Density-based Scoring: After training, the model uses a $k$-Nearest Neighbors ($k$NN) approach in the nuisance-free feature space. High distance from the training set indicates a true semantic anomaly.

Experimental Setup and Datasets

The framework was evaluated on three multi-attribute datasets: Cars3D (car models vs. poses), SmallNORB (object categories vs. lighting/elevation), and Edges2Shoes (shoe types vs. image style).

The Edges2Shoes dataset demonstrates the challenge of disentangling semantic categories (Boots, Sandals, Shoes) from nuisance factors like image style (Photos vs. Sketches).

To rigorously test the model, the authors introduced a "pseudo-anomaly" benchmark. By removing specific attribute combinations from the training set (e.g., removing all sketches of boots), they created scenarios where unseen nuisance combinations (a boot sketch) should be treated as normal, while entirely new semantic classes remain anomalies.

Key Results and Insights

Red PANDA consistently outperformed state-of-the-art methods like CSI, DN2, and MeanShifted across multiple metrics:

  • Accuracy: On Cars3D, Red PANDA achieved an AUC of 0.985, surpassing the next best method (DN2 at 0.946).
  • Nuisance Invariance (PA-Score): Red PANDA successfully recognized "pseudo-anomalies" as normal, resulting in a significantly lower false-alarm rate compared to competitors.
  • Relative Abnormality (RA-Score): The model effectively ranked true semantic anomalies higher than unseen nuisance variations.

By using disentanglement as a preprocessing filter for discriminative tasks, Red PANDA increases the signal-to-noise ratio for anomaly detection. Furthermore, by explicitly removing attributes like gender or race from the feature space, it provides a technical mechanism to reduce algorithmic bias against statistical minorities.

Limitations and Future Work

Red PANDA's primary limitation is its dependency on nuisance labels during training. While zero-shot models like CLIP can potentially automate this labeling, the system's performance is inherently tied to label quality. Additionally, the method assumes some overlap in the distribution of relevant attributes across nuisance domains. Future work will focus on applying these techniques to complex, "in-the-wild" data where nuisance factors are more difficult to disentangle than in synthetic benchmarks.

Summary updated

Abstract

Anomaly detection methods strive to discover patterns that differ from the norm in a semantic way. This goal is ambiguous as a data point differing from the norm by an attribute e.g., age, race or gender, may be considered anomalous by some operators while others may consider this attribute irrelevant. Breaking from previous research, we present a new anomaly detection method that allows operators to exclude an attribute from being considered as relevant for anomaly detection. Our approach then learns representations which do not contain information over the nuisance attributes. Anomaly scoring is performed using a density-based approach. Importantly, our approach does not require specifying the attributes that are relevant for detecting anomalies, which is typically impossible in anomaly detection, but only attributes to ignore. An empirical investigation is presented verifying the effectiveness of our approach.
BibTeX
@misc{cohen2022red,
  author = {Niv Cohen and Jonathan Kahana and Yedid Hoshen},
  title = {{Red PANDA: Disambiguating Anomaly Detection by Removing Nuisance Factors}},
  year = {2022},
  eprint = {2207.03478},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2207.03478},
}
Figure