Abstract
Autoencoder-based methods are the cornerstone of deep unsupervised outlier detection. However, these methods often underperform on complex image datasets and are sensitive to noise introduced by outliers during training. We present the Transformation Invariant AutoEncoder (TIAE), a framework that improves representation learning by requiring the model to reconstruct original images from transformed inputs. To further mitigate the impact of outliers in the training set, we incorporate an adaptive self-paced learning (SPL) strategy. Our evaluations show that TIAE advances unsupervised outlier detection performance by up to 10% AUROC compared to state-of-the-art autoencoder-based methods across five benchmark image datasets.
Introduction
Outlier detection (OD) involves identifying data points that deviate significantly from the majority of the data. In many real-world scenarios, such as fraud detection or medical diagnosis, labeled data is scarce, necessitating unsupervised approaches. The standard paradigm for unsupervised OD is the Autoencoder (AE). As shown below, a standard AE learns to compress an input $x$ into a latent representation and then reconstruct it. The underlying assumption is that the model, trained primarily on "normal" data, will fail to reconstruct outliers accurately, resulting in a high reconstruction error that serves as an anomaly score.
However, standard AEs face two major challenges. First, they often focus on low-level pixel correlations rather than high-level semantics, which is insufficient for complex datasets like CIFAR-10. Second, if the training data contains outliers, the AE may learn to reconstruct them, thereby reducing the detection sensitivity.
The TIAE Framework
To address these challenges, we propose the Transformation Invariant AutoEncoder (TIAE).
Transformation Invariant Representation Learning
The core innovation of TIAE is the shift from simple reconstruction to transformation-invariant reconstruction. As illustrated in the diagram below, the input $x$ undergoes a transformation $t(\cdot)$, such as rotation or color distortion. The encoder $E$ maps the transformed image $t(x)$ to a latent space, and the decoder $D$ must then reconstruct the original image $x$.
This approach forces the network to capture the semantic "essence" of the object (e.g., "this is a cat") to successfully undo the transformation. By focusing on high-level structural information rather than pixel-wise identity, the model becomes much more effective at distinguishing between complex inlier classes and outliers.
Adaptive Self-Paced Learning
To handle outliers present in the training data, we integrate an adaptive self-paced learning (SPL) module. SPL is a curriculum learning strategy where the model initially trains on "easy" samples (those with low reconstruction error) and gradually includes "harder" ones.
In TIAE, we introduce an adaptive threshold ($\lambda$) that is automatically calculated in each epoch based on the mean and standard deviation of the reconstruction scores. This ensures that likely outliers—which naturally yield higher errors—are excluded from the gradient updates, preventing them from "polluting" the learned representation.
Experimental Results
We evaluated TIAE on five benchmark datasets: MNIST, Fashion-MNIST, SVHN, CIFAR-10, and CIFAR-100. We employed a "one-vs-rest" protocol, where one class is designated as the inlier and the remaining classes as outliers.
The results demonstrate the superiority of the TIAE framework:
- Performance Gains: On CIFAR-10, TIAE achieved an average AUROC of 71.18%, a significant improvement over the standard Convolutional AE (55.86%) and the previous state-of-the-art, RSRAE (54.77%).
- Robustness: TIAE maintained high performance even as the outlier ratio in the training set increased from 5% to 25%, whereas baseline models showed significant degradation.
- Stability: The adaptive SPL module was found to significantly stabilize the AUROC across training epochs compared to versions of the model without the SPL component.
Conclusion
The Transformation Invariant AutoEncoder (TIAE) provides a robust and effective framework for unsupervised outlier detection. By leveraging transformation-based pretext tasks and adaptive self-paced learning, TIAE moves beyond simple pixel reconstruction to learn meaningful semantic features. This approach not only improves detection accuracy on complex datasets but also ensures stability in the presence of noisy training data. Future work may explore the application of these principles to video-based anomaly detection.
Summary updated