What do we learn? Debunking the Myth of Unsupervised Outlier Detection

Cosmin I. Bercea, Daniel Rueckert, Julia A. SchnabelarXiv 2022 · DOI · arXiv

Introduction: The Manifold Assumption Myth

In unsupervised outlier detection, the prevailing strategy relies on the Manifold Assumption. The theory suggests that an auto-encoder (AE) trained exclusively on "normal" data (e.g., healthy chest X-rays) will learn a compact representation of that specific distribution. Consequently, when presented with an anomaly—such as a lung tumor or a completely different image class—the AE should fail to reconstruct the outlier, yielding a high reconstruction error that serves as a detection signal.

However, this paper argues that this assumption is often a myth. In practice, standard AEs frequently suffer from a "Curse of Resolution." If the latent space is too large, the model simply learns an identity mapping, reconstructing anomalies perfectly. Conversely, if too constrained, it produces blurry reconstructions even for in-distribution data.

Expectation vs. Reality of Auto-Encoder reconstructions for Out-of-Distribution detection.

MorphAEus: Deformable Auto-Encoders

To address these reliability issues, the authors introduce MorphAEus, a novel deformable auto-encoder architecture. Instead of direct pixel reconstruction from a bottleneck, MorphAEus decouples the synthesis process into two distinct components:

  1. Global Image Prior: The network learns a perceptually consistent "average" representation of the training distribution. By utilizing a Perceptual Loss (PL), the model ensures that the generated prior maintains the visual characteristics of a real image rather than a blurry approximation.
  2. Local Deformation Field ($\Phi$): A secondary branch estimates a dense deformation field that "warps" the global prior to match the specific anatomical structure (morphometry) of the input image.
The MorphAEus architecture: combining a global image prior with a local deformation field.

By separating content (the prior) from shape (the deformation), MorphAEus can adapt to a patient's unique anatomy without "hallucinating" or copying abnormal textures, such as COVID-19 opacities, which were absent from the training set.

Experimental Evaluation

The model was evaluated on several benchmarks to test both domain-level and pathology-level detection:

  • MedNIST: Used for Out-of-Distribution (OoD) tasks across six medical imaging classes (e.g., training on CXR to detect Hand images).
  • MNIST: A baseline for non-medical outliers.
  • Lung Pathology Datasets: Specifically curated sets for COVID-19 and Lung Opacity to test the localization of specific diseases within the same domain.

Key Insights and Results

The authors identify a "Copy Trap" where standard Spatial AEs reconstruct unseen objects (like trucks or faces) with surprising accuracy when trained on medical images, as they learn low-level features common to all natural images.

MorphAEus demonstrates significant improvements:

  • OoD Detection: It achieved near-perfect AUROC scores (0.993 to 0.999) on MedNIST, outperforming standard VAEs and DAEs.
  • Pathology Localization: On the COVID-19 detection task, MorphAEus reached an AUROC of 0.868, a stark contrast to the 0.559 achieved by standard Spatial AEs.
  • Qualitative Success: Visually, MorphAEus generates "pseudo-healthy" reconstructions. It effectively "cleans" the input image by removing anomalies like tumors or medical devices, making the residual (the difference between input and reconstruction) a highly effective heatmap for pathology.

Limitations and Future Work

While successful, the model currently operates at 128x128 resolution, which may miss subtle pathologies. Additionally, the model is highly sensitive and may flag non-clinical anomalies such as text annotations or pacemakers. Future work involves integrating classification heads to distinguish between clinically relevant diseases and irrelevant artifacts.

Summary updated

Abstract

Even though auto-encoders (AEs) have the desirable property of learning compact representations without labels and have been widely applied to out-of-distribution (OoD) detection, they are generally still poorly understood and are used incorrectly in detecting outliers where the normal and abnormal distributions are strongly overlapping. In general, the learned manifold is assumed to contain key information that is only important for describing samples within the training distribution, and that the reconstruction of outliers leads to high residual errors. However, recent work suggests that AEs are likely to be even better at reconstructing some types of OoD samples. In this work, we challenge this assumption and investigate what auto-encoders actually learn when they are posed to solve two different tasks. First, we propose two metrics based on the Fréchet inception distance (FID) and confidence scores of a trained classifier to assess whether AEs can learn the training distribution and reliably recognize samples from other domains. Second, we investigate whether AEs are able to synthesize normal images from samples with abnormal regions, on a more challenging lung pathology detection task. We have found that state-of-the-art (SOTA) AEs are either unable to constrain the latent manifold and allow reconstruction of abnormal patterns, or they are failing to accurately restore the inputs from their latent distribution, resulting in blurred or misaligned reconstructions. We propose novel deformable auto-encoders (MorphAEus) to learn perceptually aware global image priors and locally adapt their morphometry based on estimated dense deformation fields. We demonstrate superior performance over unsupervised methods in detecting OoD and pathology.
BibTeX
@misc{bercea2022what,
  author = {Cosmin I. Bercea and Daniel Rueckert and Julia A. Schnabel},
  title = {{What do we learn? Debunking the Myth of Unsupervised Outlier Detection}},
  year = {2022},
  eprint = {2206.03698},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2206.03698},
}
Figure