Abstract
Anomaly detection (AD) typically identifies any deviation from a training distribution as an outlier. However, many variations—such as lighting, pose, or style—are semantically irrelevant "nuisance factors" that lead to false positives. We present Red PANDA (REpresentation Disentanglement for Pretrained Anomaly Detection Adaptation), a framework that utilizes domain-supervised disentanglement to filter out these specified nuisances. By learning to ignore irrelevant attributes, Red PANDA significantly improves the precision of semantic anomaly detection and mitigates algorithmic bias.
Introduction
In machine learning, the definition of an "anomaly" is often subjective. A standard detector trained on a dataset where specific attributes are correlated (e.g., all taxi drivers are female) might flag a male taxi driver as an anomaly. While statistically an outlier, such a case is semantically normal. This is the nuisance factor problem. Current AD methods struggle to distinguish between true semantic anomalies (e.g., a car driving the wrong way) and nuisance variations (e.g., a change in camera angle or style).
The authors propose Negative Attribute Guided Anomaly Detection (NAGAD), a paradigm where users specify attributes the model should ignore. This approach acknowledges that while defining every possible anomaly is impossible, identifying irrelevant variations is often straightforward for domain experts.
Methods
Red PANDA adapts pretrained representations through a three-stage process:
- Contrastive Disentanglement: Using an encoder initialized with ImageNet weights, the model applies a specialized contrastive loss to group training samples by nuisance labels. It trains the encoder to produce representations invariant to these labels, effectively "un-seeing" the nuisance attributes.
- Conditional Reconstruction: To prevent the loss of relevant semantic information, a conditional generator must reconstruct the original image using the "clean" representation combined with the nuisance label. This ensures the representation remains rich in semantic content.
- Density-based Scoring: After training, the model uses a $k$-Nearest Neighbors ($k$NN) approach in the nuisance-free feature space. High distance from the training set indicates a true semantic anomaly.
Experimental Setup and Datasets
The framework was evaluated on three multi-attribute datasets: Cars3D (car models vs. poses), SmallNORB (object categories vs. lighting/elevation), and Edges2Shoes (shoe types vs. image style).
To rigorously test the model, the authors introduced a "pseudo-anomaly" benchmark. By removing specific attribute combinations from the training set (e.g., removing all sketches of boots), they created scenarios where unseen nuisance combinations (a boot sketch) should be treated as normal, while entirely new semantic classes remain anomalies.
Key Results and Insights
Red PANDA consistently outperformed state-of-the-art methods like CSI, DN2, and MeanShifted across multiple metrics:
- Accuracy: On Cars3D, Red PANDA achieved an AUC of 0.985, surpassing the next best method (DN2 at 0.946).
- Nuisance Invariance (PA-Score): Red PANDA successfully recognized "pseudo-anomalies" as normal, resulting in a significantly lower false-alarm rate compared to competitors.
- Relative Abnormality (RA-Score): The model effectively ranked true semantic anomalies higher than unseen nuisance variations.
By using disentanglement as a preprocessing filter for discriminative tasks, Red PANDA increases the signal-to-noise ratio for anomaly detection. Furthermore, by explicitly removing attributes like gender or race from the feature space, it provides a technical mechanism to reduce algorithmic bias against statistical minorities.
Limitations and Future Work
Red PANDA's primary limitation is its dependency on nuisance labels during training. While zero-shot models like CLIP can potentially automate this labeling, the system's performance is inherently tied to label quality. Additionally, the method assumes some overlap in the distribution of relevant attributes across nuisance domains. Future work will focus on applying these techniques to complex, "in-the-wild" data where nuisance factors are more difficult to disentangle than in synthetic benchmarks.
Summary updated