Learning Transferable Representations for Image Anomaly Localization Using Dense Pretraining

Haitian He, Sarah M. Erfani, Mingming Gong, Qiuhong KeWACV 2024 · DOI

Image Anomaly Localization (IAL) is often described as the "needle in a haystack" problem of computer vision. The goal is to identify the exact pixels representing a defect—such as a hairline scratch on a metal surface or a missing component on a circuit board—without having seen that specific defect during training. While current methods show promise, they often struggle with real-world transferability and spatial precision.

Researchers have introduced DS2 (Dual Scale Dual Similarity), a dense pretraining method designed to learn generic, pixel-level representations. By shifting away from synthetic defect simulation and focusing on dense self-supervised learning, DS2 achieves state-of-the-art performance while remaining highly transferable to unseen datasets.

The Granularity Mismatch in Anomaly Detection

Modern Image Anomaly Localization faces two primary hurdles. First is the Simulation Gap: many top models rely on "outlier exposure," training on synthetic anomalies. While effective on specific benchmarks, these models often become biased toward the specific fake defects they encountered, failing when faced with novel real-world textures.

Second is the Granularity Mismatch. Most self-supervised learning (SSL) methods operate at the instance level, treating the entire image as a single entity. However, IAL is fundamentally a dense prediction task.

Figure 1: Comparison between (a) instance-level pretraining, which pools spatial information into a single vector, and (b) dense-level pretraining, which preserves spatial features for localized loss calculation.

As illustrated in Figure 1, pooling spatial information into a single vector causes the model to lose the fine-grained nuances required to distinguish a tiny defect from a normal background texture. Dense pretraining, by contrast, maintains the spatial integrity of the feature map, making it inherently more compatible with localization tasks.

The DS2 Framework

The DS2 framework avoids synthetic anomalies entirely, opting for a two-stage approach centered on robust representation learning.

Stage 1: Dense Pretraining

Building upon the PixPro architecture, DS2 introduces three critical refinements tailored for anomaly detection:

  1. Highly Selective Positive Pairs: Standard SSL considers regions similar if they are in general proximity. DS2 enforces a much tighter geometric threshold, forcing the model to learn distinct representations for different parts of an object (e.g., the head vs. the thread of a screw).
  2. Feature-wise Top-1 Similarity: To maintain global context, the model identifies the most similar feature pair across the entire image, regardless of distance. This ensures semantic consistency across uniform textures.
  3. Multi-scale Learning: By extracting features from multiple backbone layers (specifically conv4 and conv5), the model simultaneously captures fine-grained details and high-level structural information.
Figure 3: The DS2 architecture overview, showing the dual-branch backbone, multi-scale feature extraction (conv4 and conv5), and the integration of geometric proximity and top-1 similarity for loss calculation.

Stage 2: Anomaly Scoring

Once the model is pretrained, the backbone is frozen. A simple Gaussian Density Estimator (GDE) learns the distribution of "normal" features from a clean training set. During inference, any feature that deviates significantly from this learned distribution is flagged as an anomaly.

Key Results and Transferability

DS2 was evaluated against major industrial benchmarks, including MVTec AD, MVTec LOCO, KSDD2, and MTD. The results highlight several key advantages:

  • Superior Accuracy: On the MVTec AD benchmark, DS2 achieved a localization score of 94.6 AUC, outperforming other two-stage self-supervised models like SimCLR and CutPaste in a "one-for-all" setting.
  • The "One-for-All" Advantage: Unlike many competitors that require a separate model for every object category (e.g., one for bottles, one for cables), DS2 can be trained as a single holistic model for all categories, significantly simplifying real-world deployment.
  • Strong Transferability: When tested on completely new datasets (KSDD2 and MTD) without any retraining, DS2 significantly outperformed end-to-end models. This suggests the representations it learns are truly generic and not overfitted to a specific dataset's characteristics.
  • Efficiency: By focusing on representation quality rather than complex architectures, DS2 converges in roughly 9 hours, compared to over 32 hours for previous state-of-the-art two-stage models.

Limitations and Future Outlook

Despite its strengths, DS2 is not without limitations. The model can be sensitive to image noise, occasionally mistaking random perturbations for defects. Furthermore, while it excels at finding "alien" patterns like cracks or scratches, it still faces challenges with subtle structural deformations in complex, repeating textures, such as a slightly warped grid. Future work will likely explore specialized self-supervised tasks to further sharpen the model's ability to discriminate between normal structural variations and actual defects.

Summary updated

BibTeX
@inproceedings{he2024learning,
  author = {Haitian He and Sarah M. Erfani and Mingming Gong and Qiuhong Ke},
  title = {{Learning Transferable Representations for Image Anomaly Localization Using Dense Pretraining}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00115},
}
Figure