Mean-Shifted Contrastive Loss for Anomaly Detection

Tal Reiss, Yedid Hoshen

Introduction

Anomaly detection (AD) is a critical task in AI safety and industrial quality control, focused on identifying samples that deviate from a "normal" training distribution. A common and effective paradigm involves fine-tuning a model pre-trained on a large-scale dataset (like ImageNet) on the specific "normal" data of interest. However, recent research has uncovered a surprising limitation: standard contrastive loss, the state-of-the-art for self-supervised representation learning, often degrades performance when used for AD fine-tuning.

The Challenge: Uniformity vs. Compactness

The primary goal of standard contrastive learning is to achieve uniformity—spreading feature representations evenly across a unit hypersphere to maximize discriminability. In contrast, effective anomaly detection relies on compactness, where normal data points are tightly clustered.

When standard contrastive loss is applied to pre-trained features, it tends to "blow up" these tight clusters to satisfy the uniformity objective. This dispersion makes it significantly harder to distinguish normal data from anomalies, as the boundary between the two becomes blurred.

Training dynamics comparing standard and mean-shifted contrastive loss. (a) shows average cosine similarity to the training set, while (b) and (c) show similarity to augmentations for standard and MSC loss respectively.

Methodology: Mean-Shifted Contrastive Loss

To resolve this tension, the authors introduce the Mean-Shifted Contrastive (MSC) loss. The core innovation is a simple coordinate shift: instead of calculating the contrastive loss relative to the origin, the objective is centered around the mean (center) of the pre-trained features of the normal data.

Conceptual comparison between standard contrastive loss and mean-shifted contrastive loss. By shifting the center to C, MSC avoids pushing normal samples away from their own cluster center.

By shifting the reference point to the data's center, the optimization no longer forces the normal data to spread out across the entire sphere. Instead, it encourages the model to learn discriminative features within the local neighborhood of the normal class. Additionally, the authors incorporate an angular center loss to further stabilize the cluster and prevent feature collapse.

Experimental Results

The MSC loss achieves state-of-the-art results across several benchmarks:

  • CIFAR-10 & CIFAR-100: Reached 98.6% and 96.4% ROC-AUC respectively, outperforming specialized methods like PANDA and CSI.
  • Small Data Efficiency: MSC is remarkably robust in low-data regimes, maintaining high performance (96.5% on CIFAR-10) with as few as 200 training images.
  • Generalization: The method proved effective across diverse datasets, including MVTec AD (industrial inspection) and DIOR (aerial imagery), and across various architectures like ResNet and Vision Transformers (ViT).

Discussion and Limitations

While MSC significantly improves AD performance, it is not without challenges. Like many one-class classification methods, it can suffer from "catastrophic collapse" if trained for too long, necessitating careful early stopping. Furthermore, its success depends on the quality of the initial pre-trained features. Future work may focus on more robust regularization techniques to mitigate collapse and exploring optimal pre-training sources for niche domains.

Summary updated

BibTeX
@inproceedings{reiss2023mean,
  author = {Tal Reiss and Yedid Hoshen},
  title = {{Mean-Shifted Contrastive Loss for Anomaly Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i2.25309},
}
Figure