Abstract
Self-supervised anomaly detection (SSAD) relies heavily on the quality of synthetic augmentations used to simulate anomalies. However, selecting the optimal hyperparameters for these augmentations remains a significant challenge in unsupervised settings where ground-truth labels are unavailable. We present Discordance and Separability Validation (DSV), a novel unsupervised validation loss designed to identify high-performing models by measuring the geometric alignment between synthetic augmentations and real-world anomalies. Evaluated across 21 industrial inspection tasks, DSV consistently outperforms existing model selection baselines, providing a robust framework for hyperparameter optimization in SSAD.
Introduction
Self-supervised anomaly detection has emerged as a powerful paradigm for identifying outliers in unlabeled datasets. The standard approach involves training a model to distinguish between "normal" data and "pseudo-anomalies" generated via controlled transformations, such as rotations, cutouts, or color shifts. While effective, the performance of these models is highly sensitive to the choice of augmentation hyperparameters (HPs). For instance, an augmentation that is too subtle may fail to provide a useful training signal, while one that is too aggressive may destroy the underlying features of the normal class.
In supervised learning, hyperparameter selection is typically managed via a labeled validation set. In the context of anomaly detection, such labels are rarely available, leading many practitioners to rely on default settings or manual "cherry-picking." This paper addresses the Unsupervised Outlier Model Selection (UOMS) problem by introducing a validation loss that identifies the most effective augmentation strategies without requiring labeled data.
The DSV Framework: Measuring Alignment
The core contribution of this work is the concept of Alignment. An augmentation function is considered well-aligned if the pseudo-anomalies it generates occupy the same region of the embedding space as the actual (but unknown) anomalies in the test set. To quantify this without labels, the authors decompose alignment into two geometric components within the model's embedding space:
- Discordance ($h_d$): This measures the degree to which test anomalies deviate from the direct path between normal data and synthetic augmented data. High discordance suggests that the synthetic anomalies are not representative of the true outliers.
- Separability ($h_s$): This measures the distance between the normal data and the test anomalies along the direction defined by the augmentation. Higher separability indicates that the model can clearly distinguish between the two distributions.
Surrogate Losses
Since the identity of anomalies in the test set is unknown, DSV utilizes the entire unlabeled test set as a proxy through two surrogate losses:
- Discordance Surrogate ($\mathcal{L}_{dis}$): Leveraging the triangle inequality, this loss estimates how well the augmented data represents the potential distribution of outliers within the unlabeled pool.
- Separability Surrogate ($\mathcal{L}_{sep}$): This term measures the "scatteredness" of the test data along the augmentation vector, identifying configurations where the model effectively pushes potential anomalies away from the normal cluster.
The final DSV loss is a combination of these terms. By minimizing DSV, practitioners can select hyperparameters that maximize the likelihood of high detection accuracy on a specific, unseen dataset.
Experimental Evaluation
The efficacy of DSV was tested on 21 real-world tasks derived from two prominent computer vision benchmarks:
- MVTec AD: 15 tasks focusing on industrial defect detection (e.g., cracks, smudges, or structural flaws in materials like leather, cable, and glass).
- MPDD: 6 tasks involving painted metal parts with complex, non-homogeneous backgrounds.
The researchers evaluated DSV against four common augmentation types—CutOut, CutAvg, CutDiff, and CutPaste—and compared its performance against eight established unsupervised model selection baselines, including Maximum Mean Discrepancy (MMD) and HITS.
Key Findings
- Superior Performance: DSV consistently outperformed all baselines. On average, selecting models via DSV resulted in up to a 12.2% increase in AUC compared to a baseline of averaging all candidate models.
- Statistical Significance: Using the Wilcoxon signed-rank test, the improvements provided by DSV were found to be statistically significant ($p < 0.001$) across the breadth of the benchmarks.
- The "No-Free-Lunch" Observation: The experiments confirmed that optimal hyperparameters are highly dataset-specific. For example, the ideal augmentation size for detecting defects in "Carpet" differs significantly from that for "Screw," highlighting the necessity of a data-driven selection tool like DSV.
- Reliability: DSV demonstrated a strong negative correlation with actual AUC, serving as a dependable "compass" for model selection where other internal measures fluctuated unpredictably.
Discussion and Future Work
While DSV represents a significant step forward for UOMS in self-supervised anomaly detection, several avenues for future research remain. The current implementation relies on Euclidean distance in the embedding space; exploring alternative metrics, such as Chebyshev or Mahalanobis distances, might better capture specific types of anomalous alignment. Additionally, the method assumes that the underlying encoder has learned a reasonably meaningful representation of the data. If the encoder is poorly initialized, the geometric assumptions underlying the discordance and separability measures may lose their validity.
Finally, while this study focused on image data, the theoretical framework of DSV is generalizable. Future applications could extend this geometric alignment approach to tabular data or time-series analysis, where hyperparameter sensitivity is equally prevalent but less understood.
Summary updated