This paper presents a novel approach to unsupervised anomaly detection, addressing the common pitfalls of generative models that often fail to distinguish between "normal" data and out-of-distribution (OOD) samples.
Introduction
Anomaly detection (AD) is the task of identifying samples that do not belong to the distribution of the training data. This is critical for safety-sensitive applications like self-driving cars or medical diagnostics. However, existing deep generative approaches suffer from two major failures:
- Likelihood Paradox: High-capacity generative models (like Flow-based models) often assign higher likelihoods to OOD data (e.g., simple digits) than to the complex data they were trained on (e.g., natural images).
- Semantic Drift in Autoencoders: Standard Adversarial Autoencoders (AAEs) focus on making reconstructions look "realistic" but often fail to make them look like the original input. For example, an AAE trained on various animals might reconstruct a cat as a dog; while the dog looks "real," the reconstruction is semantically incorrect.
Methodology: The AMA Framework
The authors introduce Adversarial Mirrored Autoencoders (AMA), which combine a "mirrored" Wasserstein loss for semantically consistent reconstructions with a novel latent space regularization technique.
Key Innovations
- Mirrored Wasserstein Loss: Unlike standard GAN-based losses that compare the distribution of inputs to the distribution of reconstructions, the mirrored loss looks at the joint distribution of $(input, input)$ versus $(input, reconstruction)$. This forces the discriminator to ensure that the reconstruction is not just a valid sample from the training set, but a faithful reproduction of that specific input.
- Latent Space Regularization:
- Simplex Interpolation: The model enforces that linear combinations of multiple latent codes must also decode into realistic images, smoothing the latent manifold.
- Atypical Selection: Leveraging the "Gaussian Annulus Theorem," the authors sample synthetic negative points just inside or outside the high-dimensional shell where data typically resides. Training the model to "reject" these atypical points helps define a tighter boundary for normal data.
- The A-score: Instead of using raw reconstruction error, the authors fit a Gaussian distribution to the reconstruction scores $R(x)$ of the training data. The final anomaly score (A-score) is the likelihood of a test sample's error under this distribution, measuring how "typical" a sample's reconstruction error is.
Experimental Results
The authors evaluated AMA across a wide range of standard computer vision benchmarks, including CIFAR-10, SVHN, and FashionMNIST as in-distribution data, and ImageNet, LSUN, and MNIST as anomalies.
AMA consistently outperformed existing state-of-the-art (SOTA) methods. For instance, on the CIFAR-10 vs. SVHN task, AMA achieved an AUROC of 0.958, significantly higher than DeepSVDD (0.533) and Likelihood-ratio methods (0.931).
Ablation Insights
Ablation studies (shown in the table above) revealed that the Mirrored Wasserstein Loss provided the single largest boost in performance. This confirms that maintaining semantic consistency between input and reconstruction is vital for effective anomaly detection. While AMA performs excellently on distinct datasets, it still finds "near-OOD" tasks challenging, such as distinguishing CIFAR-10 from CIFAR-100.
Conclusion
By addressing semantic drift through mirrored losses and leveraging high-dimensional geometry for latent regularization, AMA provides a robust framework for unsupervised anomaly detection. The transition from measuring error magnitude to measuring error typicality (via the A-score) represents a significant step forward in making generative models reliable for OOD detection.
Summary updated