Abstract
Deep learning classifiers often exhibit overconfidence when presented with out-of-distribution (OOD) samples, posing significant risks in safety-critical applications. This paper introduces QuantOD, a novel three-stage framework for unsupervised outlier detection. QuantOD trains a normalizing flow on high-level feature representations extracted from a pre-trained classifier using a unique quantile-based maximum likelihood objective. By prioritizing the learning of "hard" inliers at the distribution's boundary, the model achieves state-of-the-art performance in separating known classes from anomalies without requiring exposure to outlier data during training.
Introduction
A critical failure mode in modern deep learning is the inability to "know what it doesn't know." Classifiers frequently assign high-confidence scores to images from unknown categories. Existing solutions typically rely on supervised training with synthetic outliers or unsupervised generative models operating in pixel space. However, pixel-based models often focus on low-level statistics rather than semantic content, leading to poor OOD detection. QuantOD addresses these challenges by leveraging semantic features and a specialized loss function to define a robust boundary for normal data.
Methodology
The QuantOD framework operates in three distinct stages to build a robust density estimator for outlier detection:
- Discriminative Training: A standard backbone (e.g., WideResNet) is trained on the inlier dataset to learn discriminative semantic features.
- Generative Training: High-level features are extracted from the classifier's penultimate layer. A Normalizing Flow (Glow architecture) is then trained to map these features to a standard normal distribution. The core innovation is the Quantile Loss ($L_{qNLL}$), which minimizes the $q$-quantile (e.g., bottom 5%) of log-likelihoods in a batch. This forces the model to focus on "borderline" inliers, sharpening the decision boundary.
- Model Inference: During testing, the Normalizing Flow calculates a log-likelihood score for new samples. Samples falling below a calibrated threshold $\tau$ are flagged as outliers.
Experimental Results
The authors evaluated QuantOD using CIFAR-10 and CIFAR-100 as inlier sets against six diverse outlier datasets: Textures (T), SVHN (S), Places365 (P), LSUN-C (LC), LSUN-R (LR), and iSUN (iS).
Performance Gains
QuantOD significantly outperforms existing unsupervised state-of-the-art methods. On CIFAR-10, it achieved an average False Positive Rate (at 95% TPR) of 17.17%, compared to 33.01% for Energy-based models and 37.08% for Mahalanobis distance methods. The use of the quantile loss ($q=0.05$) consistently yielded better results than standard mean negative log-likelihood (NLL) across various OOD scenarios.
Key Insights
- Semantic Features: Training on internal features rather than raw pixels allows the density estimator to inherit the classifier's semantic understanding.
- Boundary Focus: The quantile-based loss effectively models the tails of the distribution, where the distinction between inliers and outliers is most critical.
- Dimensionality: Lower-dimensional embeddings (e.g., 128-dim) were found to be more effective for density estimation than higher-dimensional ones, avoiding the sparsity issues of high-dimensional spaces.
Conclusion
QuantOD provides a powerful, unsupervised approach to outlier detection by combining discriminative feature extraction with quantile-based generative modeling. By focusing on the distribution's boundary, it achieves substantial improvements in robustness and efficiency. Future work may explore class-conditional normalizing flows to further refine the density estimation for complex, multi-class datasets.
Summary updated