Abstract
Deep Neural Networks (DNNs) often exhibit overconfidence when encountering Out-of-Distribution (OOD) data, posing significant risks in safety-critical applications. Traditional OOD detection methods typically require large labeled datasets or assume "pure" unlabeled data. This paper presents Adaptive In-Out-aware Learning (AIOL), a framework designed for realistic scenarios where unlabeled data is a mixture of In-Distribution (ID) and OOD samples. AIOL utilizes adaptive temperature scaling and a Gaussian Mixture Model (GMM) for dynamic sample selection, enabling the model to effectively distinguish between known classes, seen OOD classes from the unlabeled pool, and entirely novel unseen OOD classes. Experimental results on CIFAR benchmarks demonstrate that AIOL significantly outperforms existing baselines, particularly in low-label regimes.
Introduction
The deployment of machine learning models in the real world necessitates robust Out-of-Distribution (OOD) detection to prevent confident but incorrect predictions on unfamiliar inputs. However, current research often relies on two impractical assumptions: the availability of massive labeled In-Distribution (ID) data and the purity of unlabeled datasets. In practice, unlabeled data harvested from the wild is often a "mixed bag" containing both ID and OOD samples. Furthermore, models must generalize to "unseen" OOD classes that were not present in the training mixture.
AIOL addresses these challenges by treating OOD samples within the unlabeled pool as a valuable training signal rather than noise. By learning what "not-ID" looks like from the mixed data, the model becomes better equipped to identify novel classes during deployment.
Methodology: Adaptive In-Out-aware Learning (AIOL)
The AIOL framework is built upon three core components designed to maximize the utility of mixed unlabeled data.
Adaptive Temperature Scaling
A key theoretical contribution of this work is the justification for temperature scaling. While sharpening ($T < 1$) is common in semi-supervised learning to produce hard labels, the authors prove that softening ($T > 1$) in the softmax layer increases the confidence gap between ID and OOD samples. AIOL adaptively optimizes this temperature $T$ during training based on a small validation set to maintain optimal separation.
Dynamic Sample Selection via GMM
To handle the evolving nature of the model's confidence, AIOL avoids static thresholds. Instead, it fits a two-component Gaussian Mixture Model (GMM) to the confidence scores of the unlabeled data at every epoch. This allows the model to dynamically "self-label" unlabeled samples as either potential ID or OOD, adapting to the model's current state of learning.
Entropy Optimization and Augmentation
Once samples are categorized by the GMM:
- ID Samples: The model undergoes entropy minimization to reinforce high-confidence predictions on known classes.
- OOD Samples: The model performs entropy maximization, encouraging the network to express high uncertainty.
To improve generalization to unseen OOD classes, AIOL employs RandAugment and a modified Mixup strategy, increasing the diversity of the training signal.
Experimental Evaluation
The researchers evaluated AIOL using CIFAR-10 and CIFAR-100 as ID datasets, with various datasets (SVHN, ImageNet, etc.) serving as seen and unseen OOD benchmarks.
Key Results
- Superior Detection: On CIFAR-10 vs. SVHN, AIOL achieved a near-perfect AUROC of 100.0%, significantly higher than the 54.2% achieved by standard baselines.
- Generalization to Unseen Classes: AIOL maintained high performance even on datasets never encountered during training, such as a 96.9% AUROC on ImageNet as an unseen OOD.
- Label Efficiency: The method is highly effective in low-data scenarios. With only 250 labeled samples, AIOL achieved 99.8% AUROC on seen OOD tasks, whereas methods like FixMatch dropped to 42.1%.
Conclusion
AIOL provides a robust framework for OOD detection by embracing the complexity of real-world unlabeled data. By leveraging adaptive temperature scaling and dynamic sample selection, it effectively bridges the gap between semi-supervised learning and OOD detection. While there is a slight trade-off in ID classification accuracy compared to pure semi-supervised methods, the gains in safety and reliability through superior OOD detection make AIOL a compelling approach for real-world AI deployment.
Summary updated