MOS: Towards Scaling Out-of-Distribution Detection for Large Semantic Space

Rui Huang, Yixuan Li

Introduction

In real-world deployments, machine learning models trained on fixed categories inevitably encounter out-of-distribution (OOD) inputs. While OOD detection is well-studied for small-scale datasets like CIFAR, scaling these methods to large semantic spaces like ImageNet-1k remains a significant challenge. This paper, presented at CVPR 2021, introduces MOS (Minimum Others Score), a group-based framework designed to handle large-scale OOD detection with high accuracy and efficiency.

The Challenge of Large Semantic Spaces

Standard OOD detection methods often fail as the number of classes increases. For instance, the False Positive Rate (FPR95) of a common baseline jumps from 17.34% to 76.94% when moving from 50 to 1,000 ImageNet categories. In a large semantic space, decision boundaries become complex, making it difficult for models to distinguish between known classes and novel, unseen inputs.

Methodology: Divide and Conquer

The MOS framework addresses this by decomposing the large label space into smaller, manageable groups.

The MOS framework overview, showing feature extraction, group-based softmax with an 'others' category, and the final OOD detection logic.

Group Decomposition and the "Others" Category

The label space is divided into $K$ groups based on conceptual similarity (e.g., "animals," "plants"). A critical innovation is the addition of a virtual "others" category to each group. During training, images from one group serve as negative examples ("others") for all other groups. This allows the model to learn "out-of-group" characteristics without requiring external outlier datasets.

MOS Scoring Function

At test time, the model computes a softmax for each group. The Minimum Others Score (MOS) is calculated as the negative of the minimum probability of being an "other" across all groups. If an image is truly OOD, it will likely have a high "others" probability in every group. Conversely, an in-distribution image will have a very low "others" score in its corresponding group.

Experimental Setup and Datasets

The authors evaluated MOS using ImageNet-1k as the in-distribution (ID) dataset. To rigorously test OOD detection, they curated four high-resolution OOD datasets:

Examples of in-distribution (ImageNet-1k) and out-of-distribution (iNaturalist, SUN, Places, Textures) datasets used for evaluation.
  • iNaturalist: 110 plant classes distinct from ImageNet.
  • SUN & Places: Diverse scene categories (e.g., deserts, forests).
  • Textures: A collection of textural patterns.

Key Results and Insights

MOS significantly outperforms existing state-of-the-art methods in both accuracy and speed:

  • Reduced Error: MOS reduced the average FPR95 by 14.33% compared to KL Matching.
  • Inference Speed: The method is 6x faster than previous top-performing approaches because it avoids complex distance calculations to every class center.
  • Scalability: The performance gap between MOS and baselines widens as the number of classes increases, demonstrating its robustness at scale.

The study also highlights that while stronger backbones improve performance, fine-tuning only the final layer often yields better OOD detection than full-network fine-tuning, which can lead to overconfident (and incorrect) predictions on OOD data.

Conclusion

MOS provides a scalable and efficient solution for OOD detection in large-scale classification tasks. By leveraging group-based learning and a virtual "others" category, it simplifies decision boundaries and improves safety for real-world AI deployments. Future work may explore automated or hierarchical grouping strategies to further refine detection in even more complex environments.

Summary updated

Abstract

Detecting out-of-distribution (OOD) inputs is a central challenge for safely deploying machine learning models in the real world. Existing solutions are mainly driven by small datasets, with low resolution and very few class labels (e.g., CIFAR). As a result, OOD detection for large-scale image classification tasks remains largely unexplored. In this paper, we bridge this critical gap by proposing a group-based OOD detection framework, along with a novel OOD scoring function termed MOS. Our key idea is to decompose the large semantic space into smaller groups with similar concepts, which allows simplifying the decision boundaries between in- vs. out-of-distribution data for effective OOD detection. Our method scales substantially better for high-dimensional class space than previous approaches. We evaluate models trained on ImageNet against four carefully curated OOD datasets, spanning diverse semantics. MOS establishes state-of-the-art performance, reducing the average FPR95 by 14.33% while achieving 6x speedup in inference compared to the previous best method.
BibTeX
@inproceedings{huang2021mos,
  author = {Rui Huang and Yixuan Li},
  title = {{MOS: Towards Scaling Out-of-Distribution Detection for Large Semantic Space}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  doi = {10.1109/cvpr46437.2021.00860},
}
Figure