Introduction
In real-world deployments, machine learning models trained on fixed categories inevitably encounter out-of-distribution (OOD) inputs. While OOD detection is well-studied for small-scale datasets like CIFAR, scaling these methods to large semantic spaces like ImageNet-1k remains a significant challenge. This paper, presented at CVPR 2021, introduces MOS (Minimum Others Score), a group-based framework designed to handle large-scale OOD detection with high accuracy and efficiency.
The Challenge of Large Semantic Spaces
Standard OOD detection methods often fail as the number of classes increases. For instance, the False Positive Rate (FPR95) of a common baseline jumps from 17.34% to 76.94% when moving from 50 to 1,000 ImageNet categories. In a large semantic space, decision boundaries become complex, making it difficult for models to distinguish between known classes and novel, unseen inputs.
Methodology: Divide and Conquer
The MOS framework addresses this by decomposing the large label space into smaller, manageable groups.
Group Decomposition and the "Others" Category
The label space is divided into $K$ groups based on conceptual similarity (e.g., "animals," "plants"). A critical innovation is the addition of a virtual "others" category to each group. During training, images from one group serve as negative examples ("others") for all other groups. This allows the model to learn "out-of-group" characteristics without requiring external outlier datasets.
MOS Scoring Function
At test time, the model computes a softmax for each group. The Minimum Others Score (MOS) is calculated as the negative of the minimum probability of being an "other" across all groups. If an image is truly OOD, it will likely have a high "others" probability in every group. Conversely, an in-distribution image will have a very low "others" score in its corresponding group.
Experimental Setup and Datasets
The authors evaluated MOS using ImageNet-1k as the in-distribution (ID) dataset. To rigorously test OOD detection, they curated four high-resolution OOD datasets:
- iNaturalist: 110 plant classes distinct from ImageNet.
- SUN & Places: Diverse scene categories (e.g., deserts, forests).
- Textures: A collection of textural patterns.
Key Results and Insights
MOS significantly outperforms existing state-of-the-art methods in both accuracy and speed:
- Reduced Error: MOS reduced the average FPR95 by 14.33% compared to KL Matching.
- Inference Speed: The method is 6x faster than previous top-performing approaches because it avoids complex distance calculations to every class center.
- Scalability: The performance gap between MOS and baselines widens as the number of classes increases, demonstrating its robustness at scale.
The study also highlights that while stronger backbones improve performance, fine-tuning only the final layer often yields better OOD detection than full-network fine-tuning, which can lead to overconfident (and incorrect) predictions on OOD data.
Conclusion
MOS provides a scalable and efficient solution for OOD detection in large-scale classification tasks. By leveraging group-based learning and a virtual "others" category, it simplifies decision boundaries and improves safety for real-world AI deployments. Future work may explore automated or hierarchical grouping strategies to further refine detection in even more complex environments.
Summary updated