TL;DR
This paper introduces FUN-AD, a novel framework for anomaly detection that works even when the training data is unlabeled and contains anomalies. It leverages statistical observations about feature distances to assign pseudo-labels and a mutual smoothness loss to improve robustness, achieving state-of-the-art results on challenging industrial anomaly detection benchmarks.
What Problem Does This Paper Address?
Industrial anomaly detection is crucial for quality control, but real-world scenarios present significant challenges. Firstly, anomalies are inherently rare, leading to severe data imbalance. Secondly, training data can be "noisy," meaning it might contain mislabeled samples or even anomalies due to annotation errors or outdated product information. Traditional "one-class classification" methods, while common, still require clean, anomaly-free training data, which is often impractical to obtain. Existing fully unsupervised methods often struggle with noisy data, leading to degraded performance. This paper tackles the problem of building an accurate anomaly detection system without any labeled data, even when the training set is contaminated with anomalies.
Methods
FUN-AD is built upon two key observations about feature distributions:
- Pairwise Feature Distances: Features from normal samples are statistically more likely to be closer to each other than features from normal-to-anomaly pairs.
- Mutual Closest Pairs: Pairs of features that are mutually closest to each other are likely to belong to the same class (i.e., both normal or both anomalous).
These observations motivate the core components of FUN-AD:
- Iteratively Re-constructed Memory Bank (IRMB): Instead of relying on a fixed memory bank, FUN-AD iteratively refines a memory bank of "normal" features. It uses pairwise distance statistics to identify and retain confident normal samples, gradually filtering out anomalies from the memory bank over training iterations.
- Pseudo-Label Assignment: Based on the IRMB and the statistical analysis of pairwise distances, the framework assigns pseudo-labels to training patches. This allows the model to learn from potentially noisy data by identifying confident normal and anomalous samples.
- Mutual Smoothness Loss (LMS): To further improve robustness against incorrect pseudo-labels, a novel loss function is introduced. It encourages anomaly scores of mutually closest feature pairs (identified using a similarity metric) to be similar, leveraging the assumption that they belong to the same class.
- Gaussian Perturbation: To mitigate the impact of false positives and negatives in pseudo-labeling, ambiguous features (those close to the anomaly threshold) are augmented with Gaussian noise, acting as a form of regularization.
The overall framework combines these components with a standard cross-entropy loss for training a "Local-Net" feature extractor.
Datasets
The paper evaluates FUN-AD on two widely recognized industrial anomaly detection benchmarks:
- MVTec AD [3]: This dataset contains 15 categories (10 objects, 5 textures). For experiments, the authors artificially contaminate the training set by introducing anomalies from the test set at various ratios (e.g., 1:10).
- VisA [44]: This dataset includes 12 object categories. Similar contamination strategies were employed.
Additionally, a toy experiment was conducted on CIFAR-10 [15], using "automobile" as the normal class and other classes as anomalies, to demonstrate semantic anomaly detection.
Key Insights and Ideas
- Leveraging "Noisy" Data: The core insight is that even noisy, unlabeled data can be valuable if we can statistically infer which samples are likely normal and which are anomalous. This moves beyond the strict requirement of clean data for one-class methods.
- Pairwise Distances as a Signal: The paper analytically and empirically demonstrates that simple pairwise feature distances, under the assumption of wider anomaly variance, can provide a strong signal for distinguishing normal from anomalous data, even without explicit labels.
- Iterative Refinement is Key: The IRMB approach is crucial. By continuously updating the memory bank with likely normal features, the system becomes progressively more robust to initial noise and learns a cleaner representation of normality.
- Mutual Closest Pairs for Consistency: The mutual smoothness loss is a clever way to enforce consistency. If two features are very close, they should have similar anomaly scores, which helps to correct potentially misassigned pseudo-labels.
Key Results
FUN-AD achieves state-of-the-art (SOTA) performance on both MVTec AD and VisA datasets across various scenarios, including "no overlap" (anomalies not present in test set) and "overlap" (anomalies can be present in test set).
- On MVTec AD (no overlap), FUN-AD outperforms previous SOTA methods with image-wise AUROC / pixel-wise AUROC scores of 99.2/98.6.
- On VisA (overlap), it also shows significant improvements, achieving 95.1/99.2.
- The method demonstrates robustness to different anomaly-to-normal ratios in the training data.
- Ablation studies confirm the effectiveness of the mutual smoothness loss and Gaussian perturbation, showing significant performance drops when these components are removed.
Limitations and Future Work
- Sensitivity to Thresholds: While the paper shows robustness to variations in the anomaly thresholds ($T_n$ and $T_c$), extreme values can still lead to performance degradation, particularly if $T_c$ is too low or $T_n$ is too high.
- Synthetic Anomalies: The paper notes that while using synthetic anomalies can be beneficial when real anomalies are scarce, their quality can be degraded if generated from noisy datasets, potentially impacting performance.
- Future Work: The authors suggest further exploration into the impact of synthetic anomalies and potentially more advanced methods for pseudo-label refinement. The current framework is designed for industrial anomaly detection, and its applicability to other domains could be investigated.
Summary updated