Abstract
Deep learning-based object detectors are prone to "hallucinations," where out-of-distribution (OoD) objects are incorrectly classified as known categories with high confidence. This paper identifies two primary hurdles in addressing this issue: flawed evaluation benchmarks containing mislabeled data and the limitations of traditional post-hoc filtering. We propose a novel training-time paradigm that utilizes "proximal" OoD samples to suppress the objectness score of unknown categories. Our approach reduces hallucination errors by up to 91% in YOLO-based models and demonstrates significant robustness across various architectures, including Faster R-CNN and RT-DETR, while maintaining standard detection accuracy.
Introduction
In real-world applications such as autonomous driving and robotics, the ability of an object detector to recognize what it does not know is as critical as its ability to identify known objects. Current models often exhibit overconfidence, leading to hallucinations—predicting a known class (e.g., "pedestrian") for an unrelated OoD object (e.g., a "statue").
While existing research has largely focused on post-hoc mathematical filters to catch these errors during inference, this study argues that the problem is rooted in the training phase and the benchmarks used for evaluation. We demonstrate that current OoD benchmarks are significantly noisy, and that proactive shaping of decision boundaries during training is essential for reliable deployment.
The Benchmark Integrity Problem
A significant contribution of this work is the revelation that existing OoD detection benchmarks are fundamentally flawed. Through an automated curation pipeline using open-vocabulary models (such as YOLOE) and the FiftyOne library, we discovered that up to 13% of test images in standard benchmarks contain mislabeled data.
These "Type 1" errors—where In-Distribution (ID) objects appear in OoD test sets—artificially inflate the False Positive Rate (FPR95). Essentially, models have been penalized for correctly identifying objects that were erroneously excluded from the ground truth. By calibrating these benchmarks, we provide a more accurate reflection of model performance and show that existing filters are more effective than previously reported once the data is cleaned.
Methodology: Objectness Suppression and Proximal Fine-tuning
To address hallucinations at the source, we introduce a two-pronged training strategy:
1. Proximal OoD Fine-tuning
Rather than training exclusively on known classes, we incorporate "proximal" OoD samples—objects that are semantically or visually similar to the training data but do not belong to the target classes. For instance, if a model is trained to detect "dogs," a "wolf" serves as a proximal OoD sample. This allows the model to learn the subtle nuances that distinguish known categories from "near-miss" unknowns.
2. Objectness Suppression
Unlike standard image classifiers, object detectors utilize an "objectness score" to determine if a region of interest contains a valid object before classification occurs. We leverage this mechanism by training the model to treat proximal OoD samples as background. By suppressing the objectness score for these samples, the model learns to ignore distracting unknown objects entirely, preventing the classification head from ever making a high-confidence error.
Experimental Results
We evaluated our methodology across three distinct architectures: YOLO (one-stage), Faster R-CNN (two-stage), and RT-DETR (Transformer-based), using ID datasets like BDD-100K and PASCAL-VOC against OoD sets like OpenImages and Object365.
Hallucination Reduction
The results demonstrate a dramatic improvement in safety metrics. In YOLO models tested on the BDD-100K dataset, hallucination errors were reduced by 91%. Similar gains were observed across other architectures, with an 82% reduction for RT-DETR and a 91% reduction for Faster R-CNN.
Performance Trade-offs
Critically, this increase in reliability does not come at the expense of standard performance. The Mean Average Precision (mAP) on In-Distribution data decreased by only approximately 1%, indicating that the model retains its ability to detect known objects while becoming significantly more discerning.
Explainable AI (XAI) Insights
Using saliency maps to visualize model attention, we found that vanilla models often rely on background context (e.g., grass or sky) to identify objects, leading to hallucinations when that context appears with an unknown object. Our fine-tuning strategy forces the model to ignore background textures and focus strictly on the core discriminative features of the object, effectively "debiasing" the detection process.
Conclusion and Future Work
This research highlights the importance of benchmark integrity and the power of objectness suppression in mitigating hallucinations. By teaching models to treat proximal unknowns as background, we can create significantly safer object detection systems for high-stakes environments.
Future research will focus on automating the selection of optimal proximal samples and investigating strategies to ensure models do not become overly conservative in challenging conditions, such as low lighting or heavy occlusion. We also aim to further explore debiasing techniques to eliminate the model's reliance on misleading environmental context.
Summary updated