Introduction
Deep Neural Networks (DNNs) have achieved remarkable success in object detection, yet they possess a critical vulnerability: the inability to reliably identify when an input falls outside their training distribution. This "Out-of-Distribution" (OoD) problem is particularly acute in safety-critical applications like autonomous driving, where a confident but incorrect classification of an unknown object can lead to catastrophic failures.
Existing state-of-the-art (SOTA) solutions, such as Virtual Outlier Synthesis (VOS), often impose significant burdens on developers, requiring architectural modifications, complex retraining with specialized loss functions, or the assumption that feature distributions follow simple, single-centered Gaussian patterns. However, real-world feature spaces are frequently multi-modal and non-convex. To address these challenges, we present Box Abstraction Monitors (BAM), a post-hoc, "plug-and-play" method that provides robust OoD detection without retraining or architectural changes.
The BAM Framework
BAM shifts away from complex statistical modeling in favor of a geometric approach. It encloses the feature space of In-Distribution (ID) data using a finite union of n-dimensional boxes (intervals). This allows the monitor to represent complex, non-convex decision boundaries that a single "ball" or center point cannot capture.
The construction of a BAM monitor follows a four-step pipeline:
- Feature Extraction: Features are extracted from the penultimate layers (typically the FC1 or FC2 layers) of the pre-trained model's classification head.
- Feature Partitioning: Recognizing that a single class (e.g., "pedestrian") may have multiple distinct clusters in feature space, BAM utilizes k-means clustering to partition features into $k$ subsets.
- Abstraction Building: For each cluster, a "tight box" is generated by identifying the minimum and maximum values for every dimension of the feature vectors within that cluster.
- Box Enlargement: To prevent the monitor from being overly restrictive, the boxes are expanded until they cover a target percentage (e.g., 95%) of the training data, ensuring a high True Positive Rate (TPR).
At inference time, the monitor performs a simple check: if a detected object's feature vector falls outside all the boxes associated with its predicted class, it is flagged as OoD.
Experimental Evaluation
The effectiveness of BAM was evaluated using standard autonomous driving benchmarks. The In-Distribution (ID) datasets included BDD100K and KITTI. To test Out-of-Distribution (OoD) performance, the authors used MS-COCO, OpenImages, and a filtered version of PASCAL VOC (containing only classes not present in the ID sets).
BAM was compared against existing SOTA methods, including VOS, across various backbones such as ResNet101. The primary metric for evaluation was FPR95 (the False Positive Rate at a 95% True Positive Rate), where a lower score indicates better performance.
Results and Discussion
BAM demonstrated superior performance in both accuracy and computational efficiency:
- Detection Accuracy: BAM outperformed VOS in 16 out of 18 experimental scenarios. Notably, on the BDD100K dataset with a ResNet101 backbone, BAM reduced the FPR95 from approximately 37% to 23%—a significant improvement in the reliability of the detector.
- Real-time Efficiency: Because checking if a point resides within a box requires only simple coordinate comparisons, BAM introduces negligible overhead. On an Nvidia RTX A4000, the inference time for the KITTI dataset increased by only 0.3 milliseconds (a 0.7% overhead). Even in complex configurations, overhead remained below 1.65%, making it ideal for real-time deployment.
- Interpretability: Unlike "black-box" uncertainty scores, BAM’s boundaries are defined by explicit min/max values in the feature space. This provides engineers with a more interpretable mechanism to inspect why specific samples are rejected.
Conclusion
BAM offers a practical and powerful solution for adding OoD detection to existing object detection pipelines. By utilizing box abstractions, it captures the complexity of real-world feature distributions while maintaining the speed required for autonomous systems. Its post-hoc nature ensures that it can be integrated into production models with zero retraining effort. Future work will explore refining these abstractions to reduce coarseness and extending the framework to one-stage detectors like YOLO.
Summary updated