Fast Decision Boundary based Out-of-Distribution Detector

Litian Liu, Yao Qin

Abstract

Efficient and effective Out-of-Distribution (OOD) detection is essential for the safe deployment of AI systems. Existing feature space methods, while effective, often incur significant computational overhead due to their reliance on auxiliary models built from training features. In this paper, we propose a computationally-efficient OOD detector without using auxiliary models while still leveraging the rich information embedded in the feature space. Specifically, we detect OOD samples based on their feature distances to decision boundaries. To minimize computational cost, we introduce an efficient closed-form estimation, analytically proven to tightly lower bound the distance. Based on our estimation, we discover that In-Distribution (ID) features tend to be further from decision boundaries than OOD features. Additionally, ID and OOD samples are better separated when compared at equal deviation levels from the mean of training features. By regularizing the distances to decision boundaries based on feature deviation from the mean, we develop a hyperparameter-free, auxiliary model-free OOD detector. Our method matches or surpasses the effectiveness of state-of-the-art methods in extensive experiments while incurring negligible overhead in inference latency.

Introduction

When machine learning models are deployed in real-world environments, they inevitably encounter "Out-of-Distribution" (OOD) samples—data from classes not seen during training. For safety-critical applications like autonomous driving or medical diagnosis, a model must be able to flag these samples rather than making confident, incorrect predictions.

Existing OOD detection methods generally fall into two categories: output-space methods, which are fast but often inaccurate, and feature-space methods, which are accurate but computationally expensive due to their reliance on auxiliary models (e.g., K-Nearest Neighbors or Gaussian Mixture Models). This paper introduces the Fast Decision Boundary based Out-of-Distribution Detector (fDBD), a method that bridges this gap by providing feature-space accuracy with virtually zero inference latency overhead.

The fDBD Approach: Distance to Decision Boundaries

The core intuition behind fDBD is that In-Distribution (ID) samples—those the model has been trained to classify—tend to reside deep within the "interior" of their respective class territories, far from the decision boundaries. In contrast, OOD samples often fall closer to these boundaries where the model's classification is less certain.

Conceptual illustration of feature distances to decision boundaries and the resulting distribution separation between ID and OOD samples.

To make this intuition practical for real-time systems, the authors address two major hurdles:

  1. Computational Complexity: Calculating the exact distance to a non-convex decision boundary typically requires iterative optimization, which is too slow for inference. fDBD introduces a closed-form distance estimation—a mathematical formula that provides a tight lower bound on the distance in constant time.
  2. Efficiency: By replacing iterative processes with this analytical estimation, the authors achieved a staggering 1800x speedup, reducing distance calculation time from nearly a second to just 0.53ms per image.

The Role of Regularization

While raw distance to the decision boundary is a strong signal, it isn't perfect. OOD samples can sometimes be far from boundaries simply because they are far from the entire training data manifold. To account for this, fDBD employs Feature Deviation Regularization.

The authors observed that as features move further from the training mean (the "center" of known data), their distance to decision boundaries naturally increases. By regularizing the boundary distance based on the sample's deviation from the training mean, the detector can more effectively distinguish between "confident" ID samples and "far-out" OOD samples.

Effect of regularization on the separation between ID (CIFAR-10) and OOD (SVHN) samples.

As shown in the figure above, this regularization step significantly sharpens the separation between the ID and OOD distributions, leading to a more robust detector that requires no hyperparameter tuning.

Experimental Results

The fDBD detector was evaluated across standard benchmarks including CIFAR-10 and ImageNet-1k, using architectures ranging from ResNets to Vision Transformers (ViT).

  • Superior Accuracy: On ImageNet benchmarks using ResNet-50, fDBD reduced the False Positive Rate (FPR95) to 51.19%, outperforming standard Softmax-based methods and matching the performance of much more complex feature-space methods.
  • Negligible Latency: Unlike methods like KNN or ViM, which add noticeable milliseconds of delay, fDBD's overhead is statistically invisible during a standard forward pass.
  • Robustness: The method proved effective even under domain shift (e.g., corrupted images in CIFAR-10-C), demonstrating its reliability in noisy real-world conditions.

Conclusion

fDBD offers a compelling solution for OOD detection by leveraging the geometric properties of the feature space without the "speed tax" of auxiliary models. By combining a fast closed-form distance estimation with feature deviation regularization, it provides a hyperparameter-free, post-hoc detector that is both highly effective and ready for immediate deployment in latency-sensitive AI systems.

Summary updated

Abstract

Efficient and effective Out-of-Distribution (OOD) detection is essential for the safe deployment of AI systems. Existing feature space methods, while effective, often incur significant computational overhead due to their reliance on auxiliary models built from training features. In this paper, we propose a computationally-efficient OOD detector without using auxiliary models while still leveraging the rich information embedded in the feature space. Specifically, we detect OOD samples based on their feature distances to decision boundaries. To minimize computational cost, we introduce an efficient closed-form estimation, analytically proven to tightly lower bound the distance. Based on our estimation, we discover that In-Distribution (ID) features tend to be further from decision boundaries than OOD features. Additionally, ID and OOD samples are better separated when compared at equal deviation levels from the mean of training features. By regularizing the distances to decision boundaries based on feature deviation from the mean, we develop a hyperparameter-free, auxiliary model-free OOD detector. Our method matches or surpasses the effectiveness of state-of-the-art methods in extensive experiments while incurring negligible overhead in inference latency. Overall, our approach significantly improves the efficiency-effectiveness trade-off in OOD detection. Code is available at: https://github.com/litianliu/fDBD-OOD.
BibTeX
@inproceedings{liu2024fast,
  author = {Litian Liu and Yao Qin},
  title = {{Fast Decision Boundary based Out-of-Distribution Detector}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=eQaOb4r6YC},
}
Figure