MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space

Rui Huang, Yixuan Li

Introduction

Out-of-distribution (OOD) detection is a critical safety mechanism for machine learning models, enabling them to identify inputs that fall outside their training distribution. While existing methods perform well on small-scale datasets like CIFAR-10, they often struggle to scale to large-scale semantic spaces such as ImageNet-1k. As the number of classes increases, the decision boundary between in-distribution and out-of-distribution data becomes increasingly complex, leading to a significant rise in false positive rates. For instance, a standard baseline's error rate (FPR95) can skyrocket from ~17% to ~77% when scaling from 50 to 1,000 classes.

Methodology: The MOS Framework

To address the challenges of large-scale OOD detection, researchers have introduced MOS (Minimum Others Score), a "divide and conquer" approach that decomposes the large semantic space into smaller, manageable groups.

Overview of the MOS framework, showing group-based decomposition and the OOD detection pipeline across diverse datasets.

Group-based Decomposition

Instead of a single flat softmax over 1,000 classes, MOS partitions the label space into $K$ semantically related groups (e.g., "Animals," "Artifacts," "Plants"). Within each group, a "virtual" others category is introduced. During training, if an image belongs to one group, it is treated as an "others" instance for all other groups. This allows the model to explicitly learn the characteristics of data that does not belong to a specific semantic cluster, simplifying the decision manifold.

Scoring Function

The core of the detection mechanism is the Minimum Others Score (MOS). For a given input, the model computes a softmax probability within each group. An in-distribution image is expected to have a low "others" score in its corresponding group. Conversely, an OOD image is likely to be classified as "others" with high confidence across all groups.

Detailed view of the MOS scoring logic, where the minimum "others" score across groups determines the OOD status.

The final OOD score is determined by taking the minimum "others" probability across all groups. If this minimum score exceeds a threshold $\gamma$, the input is flagged as out-of-distribution. This logic effectively filters out anomalies that do not fit into any known semantic group.

Experimental Setup

The framework was evaluated using ImageNet-1k as the in-distribution (ID) dataset. To test OOD robustness, the authors curated four diverse, high-resolution datasets with no semantic overlap with ImageNet: iNaturalist (fine-grained plants/animals), SUN and Places (scene images), and Textures (textural patterns).

Key Results and Insights

MOS demonstrates significant improvements in both performance and efficiency:

  • Reduced Error Rates: MOS achieved a 14.33% reduction in the average FPR95 compared to previous state-of-the-art methods like KL Matching.
  • Computational Efficiency: The method provides a 6x speedup in inference time over KL Matching, making it viable for real-time applications like autonomous driving.
  • Scalability and Stability: Unlike standard softmax-based approaches, which degrade rapidly as the number of classes grows, MOS remains remarkably stable as the semantic space increases from 50 to 1,000 classes.
  • Accuracy Preservation: The group-based structure maintains high classification accuracy on the original ImageNet task (75.16% vs 75.20% for flat softmax).

Conclusion

MOS effectively scales OOD detection to large-scale datasets by simplifying the decision manifold through semantic grouping. By introducing the "others" category and the Minimum Others Score, the framework provides a robust and efficient solution for real-world deployment where large label spaces are common. While performance is currently tied to taxonomy quality (like WordNet), future work may explore dynamic or learned taxonomies to further enhance flexibility.

Summary updated

BibTeX
@inproceedings{huang2021mos,
  author = {Rui Huang and Yixuan Li},
  title = {{MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  doi = {10.1109/cvpr46437.2021.00860},
}
Figure