Box-conditional Hypothesis Testing for Out-of-Distribution Object Detection

Donghoon Lee, Jusung Park, Shinjin KangarXiv 2026 · arXiv

TL;DR

The authors introduce PLUM, a training-free method for out-of-distribution (OOD) object detection that treats the decision as a hypothesis test conditioned on bounding box geometry. By comparing detected objects only to in-distribution (ID) examples of similar size and image-border contact, PLUM achieves state-of-the-art performance without requiring retraining or external models.

What Problem Does This Paper Address?

Deep learning object detectors are notoriously overconfident. When they encounter an object from a class they weren't trained on (an OOD object), they often draw a box around it and assign it a known (ID) class with high confidence. This is a major safety concern for real-world deployments like autonomous driving.

Existing OOD detection methods often struggle because the "score" used to identify OOD objects is confounded by the bounding box's geometry. For example, a very small object or one partially cut off by the image border naturally yields different feature statistics than a large, centered one. Standard methods that apply a single global threshold across all detections suffer from "conditional bias," leading to uneven false-alarm rates—small ID objects might be wrongly flagged as OOD, while large OOD objects might slip through.

Methods

The authors propose PLUM (Pooled, Leveled, Unit-norm Mahalanobis score), which reframes OOD detection as a stratified hypothesis test.

  1. Stratification: Instead of a one-size-fits-all threshold, PLUM groups detections into "strata" based on their size level (from the feature pyramid) and whether they touch the image border.
  2. Feature Extraction at the "Task Boundary": PLUM pools features from the last backbone stage (e.g., res5 in a ResNet-50) rather than the final classification head. The logic is that the final layers are trained to discard any information not strictly necessary for identifying ID classes, potentially throwing away the very features that distinguish an unknown object.
  3. Mahalanobis Distance: It calculates the distance of a pooled, normalized feature to the nearest ID class mean, using a covariance matrix estimated from the training data.
  4. Leveling: Within each stratum, it further removes size-related trends by fitting a linear regression to the scores in log-area space.
  5. Ranking: Finally, it ranks a test box's score against a reference set of ID boxes from the same stratum. This provides a distribution-free bound on the false-alarm rate for every type of box.
Figure 2. Overview of PLUM. The detector stays frozen. PLUM pools the res5 feature inside the box from which each detection is read and measures its Mahalanobis distance to the ID class means. It then ranks this distance among ID reference detections of the same stratum. A giraffe detected as a horse with confidence 0.96 receives a PLUM value of 0.002, whereas an ID horse receives 0.92.

Datasets

The method was evaluated using standard OOD object detection benchmarks:

  • In-Distribution (ID): PASCAL-VOC and BDD100K.
  • Out-of-Distribution (OOD): Images from MS-COCO and OpenImages that contain no objects from the ID classes.
  • Detectors: Faster R-CNN (ResNet-50 FPN) and Deformable DETR.

Key Insights and Ideas

  • Geometry as a Confounder: The "null distribution" (the scores of ID objects) is not static; it shifts with box size and position. Comparing a small box to a large box is an "apples-to-oranges" comparison.
  • The Task Boundary: There is a specific point in a network where it stops being a general feature extractor and starts being a specialized ID classifier. Reading features before this boundary is critical for OOD detection.
  • Training-Free is Competitive: PLUM outperforms many methods that require complex retraining or synthetic outlier generation, proving that better statistical handling of existing features can be more effective than adding more data.
  • Stratification Improves AUROC: As shown in Figure 1, simply accounting for box size can jump AUROC from 71 to 92 in a toy example by evening out the false-alarm rates.
Figure 1. A simple example with equal numbers of small and large boxes. The distance follows a Gaussian with unit variance, with ID mean 10 on small boxes and 0 on large boxes, and OOD boxes lie 2 farther out. Each threshold keeps 95% of the ID boxes, and shading marks the part of each distribution that the test flags as OOD. One threshold for all boxes reaches an AUROC of 71, and one threshold for each size reaches 92.

Key Results

PLUM achieved the highest AUROC among compared methods that do not use external models (like CLIP):

  • PASCAL-VOC: Reached 91.5 (MS-COCO OOD) and 94.5 (OpenImages OOD) AUROC.
  • BDD100K: Reached 95.1 (MS-COCO OOD) and 97.5 (OpenImages OOD) AUROC.
  • Qualitative Success: PLUM correctly identifies OOD objects (like a zebra or a pizza) that standard thresholding might miss, while correctly keeping small or border-touching ID objects (like small airplanes or traffic signs) that standard methods might falsely reject.
Figure 3. Detections on which the two tests differ, and one failure. Each panel gives the true status of the box and the class that Faster R-CNN predicts, and the test runs at α = 5%. These detections illustrate each case and are not a sample.

Limitations and Future Work

  • Reference Set Dependency: PLUM requires a set of ID detections (from the training or validation set) to serve as a reference for ranking. While the authors show this is robust, the quality of the OOD detection is tied to how well these references represent the test-time ID distribution.
  • Backbone Reliance: Since it reads backbone features, its power is limited by what the backbone has learned. If a backbone is completely "blind" to certain visual patterns, PLUM cannot recover them.
  • Future Directions: The authors suggest extending this "box-conditional" logic to other tasks like instance segmentation (using masks) or 3D object detection. They also note that multiple testing corrections could be applied to control the error rate over all detections in an image.

Summary updated

Abstract

Detecting out-of-distribution (OOD) objects is essential for ensuring the reliability and safety of object detectors in real-world deployment. We treat the decision on a detected box as a hypothesis test with the null hypothesis that the box covers an object of an in-distribution (ID) class. The null distribution changes with the size of the box and its contact with the image border, and the detector observes both. The test can therefore rank a box only against ID boxes of the same size and border contact, which gives a distribution-free bound on the false-alarm rate for each of them and uncovers the separation that one null distribution hides. The power of the test also rests on the layer it reads. Layers built only for the ID classes have no reason to keep what sets an unknown object apart, so the test reads the features before them. We propose \textbf{PLUM}, a training-free score that pools the last backbone stage inside the box and ranks its distance to the ID classes among ID boxes of the same size and border contact. On the standard benchmark, PLUM reaches the highest AUROC among the compared methods without external models, with 91.5 and 94.5 on PASCAL-VOC and 95.1 and 97.5 on BDD100K. Code: https://github.com/DDOZMI/PLUM
BibTeX
@misc{lee2026box,
  author = {Donghoon Lee and Jusung Park and Shinjin Kang},
  title = {{Box-conditional Hypothesis Testing for Out-of-Distribution Object Detection}},
  year = {2026},
  eprint = {2611.18519},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18519},
}
Figure