Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution

Kai Liu, Zhihang Fu, Sheng Jin, Chao Chen, Ze Chen, Rongxin Jiang, Fan Zhou, Yaowu Chen, Jieping Ye

TL;DR

Most out-of-distribution (OOD) detection methods fail when training data is imbalanced, often mistaking rare "tail" classes for outliers. This paper introduces ImOOD, a statistical framework that mathematically identifies a class-aware bias in imbalanced settings and provides a plug-and-play training regularization to fix it, setting new state-of-the-art results on long-tailed benchmarks.

What Problem Does This Paper Address?

In a perfect world, machine learning models are trained on balanced datasets where every category has an equal number of examples. In the real world, data is "long-tailed": a few "head" classes have thousands of samples, while many "tail" classes have only a handful.

Standard OOD detection—the ability of a model to say "I don't know what this is"—breaks down under this imbalance. The authors identify two specific failures:

  1. Tail-class confusion: Models haven't seen enough tail-class samples, so they frequently misidentify them as OOD.
  2. Head-class "gravity": Because head classes have massive decision spaces, unknown OOD samples are often erroneously sucked into these head classes.

As shown in Figure 1, these disparate decision spaces jointly degrade performance, creating a significant gap between theoretical OOD detection and practical deployment.

Figure 1: Issues of OOD detection on imbalanced data. (a) Statistics of the class labels of ID samples that are wrongly detected as OOD, and the class predictions of OOD samples that are wrongly detected as ID. (b) Illustration of the OOD detection process in feature space. Head classes’ huge decision space and tail classes’ small decision space jointly damage the OOD detection.

Methods

The authors propose ImOOD, a generalized statistical framework to bridge the gap between balanced and imbalanced OOD detection.

Key technical contributions include:

  • Theoretical Bias Identification: They derive a "class-aware bias item" ($\beta(x)$) that mathematically explains why imbalanced training leads to poor OOD scores. They prove that the balanced posterior probability is essentially the imbalanced one multiplied by this bias term.
  • Unified Regularization: Instead of a new architecture, they introduce a training-time loss function ($L_{ood}$). This loss uses a learnable mapping ($\gamma$) to estimate the likelihood difference between balanced and imbalanced distributions.
  • Plug-and-Play Versatility: The method is designed to be integrated into existing OOD frameworks like Outlier Exposure (OE), Energy-based models, and Binary Discriminators (BinDisc). During inference, the extra terms drop away, meaning there is zero additional computational cost at test time.

Datasets

The researchers evaluated ImOOD on three primary long-tailed benchmarks:

  • CIFAR10-LT & CIFAR100-LT: Versions of the standard CIFAR datasets modified to have an imbalance ratio of 100 (the most frequent class has 100x more samples than the least).
  • ImageNet-LT: A large-scale long-tailed version of ImageNet.
  • OOD Test Sets: They used semantically coherent benchmarks including Textures, SVHN, and subsets of Places365 to test the models' ability to reject truly unknown samples.

Key Insights and Ideas

  • The "Double Whammy": Previous works mostly focused on tail-class samples being called OOD. This paper highlights that the inverse—OOD samples being called head-class ID samples—is equally damaging and requires a unified statistical fix.
  • Moving Beyond Heuristics: While earlier methods used heuristic "fixes" for imbalance, ImOOD is grounded in Bayesian principles, providing a more robust theoretical foundation for why the regularization works.
  • Input-Dependent Correction: The authors found that the bias isn't just class-dependent but also input-dependent. By making their correction factor ($\gamma$) a learnable function of the input features, they achieved better results than using a simple constant for each class.

Key Results

ImOOD consistently outperformed existing state-of-the-art (SOTA) methods across all metrics (AUROC, AUPR, and FPR95).

  • CIFAR-LT: On CIFAR10-LT, combining ImOOD with the previous SOTA (COCL) pushed AUROC to 93.55% and reduced the False Positive Rate (FPR95) to 28.52%.
  • ImageNet-LT: The improvements were even more dramatic on large-scale data. Against the strong PASCL baseline, ImOOD achieved a 6.7% increase in AUROC and a massive 13.2% decrease in FPR95.
  • Robustness: The method proved robust even when faced with "near-OOD" samples (samples very similar to the training data) and spurious correlations.

Detailed comparisons on CIFAR benchmarks are visible in Table 1: OOD detection evaluation on CIFAR10/100-LT

Table 1: OOD detection evaluation on CIFAR10/100-LT benchmarks. The best results are marked in bold, and the secondary results are marked with underlines. The base model is ResNet18.

Limitations and Future Work

The authors acknowledge a few areas for further exploration:

  • Auxiliary Data Dependency: Like many top-tier OOD methods, ImOOD currently relies on having access to some auxiliary OOD training samples to help define the decision boundary.
  • Dynamic Environments: The current framework assumes the ID/OOD boundary is static. In real-world "open-world" scenarios, new types of OOD samples might appear over time. The authors suggest that integrating online-learning strategies to adapt the boundary during testing is a promising next step.

Summary updated

Abstract

Detecting and rejecting unknown out-of-distribution (OOD) samples is critical for deployed neural networks to void unreliable predictions. In real-world scenarios, however, the efficacy of existing OOD detection methods is often impeded by the inherent imbalance of in-distribution (ID) data, which causes significant performance decline. Through statistical observations, we have identified two common challenges faced by different OOD detectors: misidentifying tail class ID samples as OOD, while erroneously predicting OOD samples as head class from ID. To explain this phenomenon, we introduce a generalized statistical framework, termed ImOOD, to formulate the OOD detection problem on imbalanced data distribution. Consequently, the theoretical analysis reveals that there exists a class-aware bias item between balanced and imbalanced OOD detection, which contributes to the performance gap. Building upon this finding, we present a unified training-time regularization technique to mitigate the bias and boost imbalanced OOD detectors across architecture designs. Our theoretically grounded method translates into consistent improvements on the representative CIFAR10-LT, CIFAR100-LT, and ImageNet-LT benchmarks against several state-of-the-art OOD detection approaches. Code is available at https://github.com/alibaba/imood.
BibTeX
@inproceedings{liu2024rethinking,
  author = {Kai Liu and Zhihang Fu and Sheng Jin and Chao Chen and Ze Chen and Rongxin Jiang and Fan Zhou and Yaowu Chen and Jieping Ye},
  title = {{Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=EWxNEnFjKR},
}
Figure