Contrastive Representations for UnsupervisedAnomaly Detection and Localization

Carsten Tim Lüth, David Zimmerer, Gregor Koehler, Paul F Jaeger, Fabian Isensee, Jens Petersen, Klaus Maier-HeinSubmission to MIDL

TL;DR

This paper introduces CRADL, a novel unsupervised anomaly detection and localization method for medical imaging. CRADL leverages contrastive learning to extract semantically rich representations from unlabeled normal scans, which are then used to train a generative model for anomaly identification. This approach outperforms existing generative models by better capturing subtle deviations from normality, achieving state-of-the-art results in anomaly localization.

What Problem Does This Paper Address?

Detecting and localizing anomalies in medical images (like tumors or lesions) is crucial for diagnosis, but it's a challenging task, especially when labeled data of anomalies is scarce. Traditional supervised methods require extensive annotations, which are costly and time-consuming to obtain. Unsupervised anomaly detection aims to overcome this by learning the distribution of "normal" data and flagging anything that deviates. However, current state-of-the-art methods, often based on generative models like VAEs and GANs, tend to focus on low-level image features (e.g., pixel intensities) rather than the richer semantic information needed to distinguish subtle anomalies. This limitation hinders their ability to accurately detect and pinpoint anomalies.

Methods

CRADL tackles this by decoupling representation learning from distribution learning. It consists of two main stages:

  1. Contrastive Pre-training: An encoder network is trained using a self-supervised contrastive learning approach, similar to SimCLR. The key idea is to train the encoder to produce similar representations for different "views" (augmentations) of the same normal image while pushing representations of different images apart. This forces the encoder to learn semantically meaningful features.
  2. Generative Model on Representations: The learned, semantically rich representations from the encoder are then used to train a generative model (e.g., a Gaussian Mixture Model or a Normalizing Flow). This generative model learns the distribution of "normal" representations. Anomalies are detected when a new image's representation falls outside this learned normal distribution. Anomaly localization is achieved by backpropagating the anomaly score from the representation space back into the image space.

Datasets

The study utilizes T2-weighted MRI brain datasets.

  • Training Dataset: A subset of the Human Connectome Project (HCP) dataset, consisting of 894 "normal" MRI scans, is used for training the encoder.
  • Anomalous Datasets: Three datasets are used for evaluation:
  • HCP Synth: An artificially generated dataset using real-world objects rendered into brain regions of HCP images, designed to maintain the original data distribution.
  • ISLES: The Ischemic Stroke Lesion Segmentation Challenge 2015 dataset, containing stroke lesions as anomalies.
  • BraTS: The Brain Tumor Segmentation Challenge 2017 dataset, featuring glioblastoma and lower-grade glioma as anomalies.

Key Insights and Ideas

  • Semantically Rich Representations are Key: The core hypothesis is that representations learned through contrastive self-supervised learning are more semantically informative than those from traditional generative models, leading to better anomaly detection and localization.
  • Decoupling Representation and Distribution Learning: CRADL separates the task of learning good representations from learning the distribution of normal data. This allows for the use of powerful contrastive methods for representation learning even without labeled anomaly data.
  • Generative Models on Representations are Efficient: Fitting generative models on low-dimensional representations is computationally much cheaper than fitting them directly on high-dimensional image space.
  • Subtle Anomalies are Better Captured: CRADL excels at detecting fine deviations from normality, particularly on datasets like HCP Synth where anomalies might differ in texture and appearance, not just intensity.

Key Results

  • Anomaly Localization: CRADL achieves state-of-the-art performance in anomaly localization, outperforming VAE and ceVAE baselines on the HCP Synth and ISLES datasets.
  • Anomaly Detection: CRADL shows competitive or superior performance in anomaly detection, particularly on the HCP Synth dataset, where it substantially outperforms VAE and ceVAE.
  • Representation Quality: The contrastive representations used by CRADL are shown to be superior to those generated by VAE and ceVAE for anomaly detection and localization tasks.
  • Model Selection: For anomaly detection, more complex generative models like Normalizing Flow on representations tend to perform best, while for anomaly localization, simpler models like GMMs are more effective.

Limitations and Future Work

  • Distribution Shifts: The performance on datasets like ISLES and BraTS, which have significant distribution shifts from the training data (due to age, scanner differences, etc.), presents a challenge. The authors hypothesize that CRADL's ability to capture fine semantic differences might be less advantageous when almost all slices could be considered anomalous due to these shifts.
  • Generalization to 3D: The current work focuses on 2D slices. Generalizing to 3D inputs would be a valuable next step.
  • Alternative Self-Supervised Tasks: Exploring other self-supervised learning tasks beyond contrastive learning could further improve performance.
  • Incorporating Restorative Techniques: Integrating image restoration or representation resampling techniques might enhance CRADL's capabilities.
  • High Anomaly Prevalence: The field generally faces the issue of high anomaly prevalence in evaluation datasets, which might bias metrics. Future work could explore evaluation with a higher proportion of normal images.

Summary updated

Abstract

Unsupervised anomaly detection in medical imaging aims to detect and localize arbitrary anomalies without requiring labels during training. Generally, this is achieved by learning a data distribution of normal samples and detecting anomalies as regions in the image which deviate from this distribution. In the medical imaging domain, most current state-of-the-art methods use latent variable generative models. Because such models operate directly on sample space, they tend to primarily encode low-level statistics (like pixel intensities), while having problems capturing fine semantic information within their representations. Recent work has shown that representations obtained from a feature extractor trained with a discriminative task are rich in semantic information. This, however, requires labeled datasets - a prerequisite that is often not fulfilled. We propose CRADL, a framework for unsupervised anomaly detection and localization consisting of a feature extractor trained with a contrastive pretext-task and a generative model which learns the distribution of representations. Through this, we circumvent the need for labels while still being able to fit the generative model on semantic-rich representations. We further compare the quality of these contrastive representations with representations obtained from a VAE and ceVAE in the context of anomaly localization. We evaluate CRADL on the BraTS and ISLES datasets, as well as an in-house dataset, and demonstrate state-of-the-art performance on the task of anomaly localization in our comparison with a VAE and ceVAE.
BibTeX
@inproceedings{luth2023contrastive,
  author = {Carsten Tim Lüth and David Zimmerer and Gregor Koehler and Paul F Jaeger and Fabian Isensee and Jens Petersen and Klaus Maier-Hein},
  title = {{Contrastive Representations for UnsupervisedAnomaly Detection and Localization}},
  booktitle = {Bildverarbeitung für die Medizin},
  year = {2023},
  doi = {10.1007/978-3-658-41657-7\_54},
}
Figure