Abstract
The availability of real-world datasets is the prerequisite for developing object detection methods for autonomous driving. While ambiguity exists in object labels due to error-prone annotation process or sensor observation noises, current object detection datasets only provide deterministic annotations without considering their uncertainty. This precludes an in-depth evaluation among different object detection methods, especially for those that explicitly model predictive probability. In this work, we propose a generative model to estimate bounding box label uncertainties from LiDAR point clouds, and define a new representation of the probabilistic bounding box through spatial distribution. Comprehensive experiments show that the proposed model represents uncertainties commonly seen in driving scenarios. Based on the spatial distribution, we further propose an extension of IoU, called the Jaccard IoU (JIoU), as a new evaluation metric that incorporates label uncertainty. Experiments on the KITTI and the Waymo Open Datasets show that JIoU is superior to IoU when evaluating probabilistic object detectors.
Introduction
In autonomous driving, training robust object detectors relies heavily on large-scale datasets like KITTI and Waymo. Traditionally, these datasets provide "ground truth" labels as deterministic bounding boxes. However, these labels are often inherently uncertain due to sensor noise and occlusions. For instance, LiDAR sensors typically only capture "L-shaped" reflections from the surfaces of a vehicle facing the sensor, leaving the far side of the object to be estimated by human annotators.
Standard evaluation metrics, such as Intersection over Union (IoU) and Average Precision (AP), ignore this ambiguity. This leads to two primary challenges: models are forced to overfit noisy labels during training, and probabilistic detectors—which output uncertainty estimates—cannot be accurately benchmarked against deterministic ground truths.
Methodology
Generative Label Uncertainty Model
To address these limitations, we propose a generative framework that treats bounding box parameters not as fixed values, but as random variables. Using Variational Bayes, we infer a posterior distribution of the bounding box parameters ($c_x, c_y, l, w, \theta$) conditioned on the observed LiDAR point cloud. This approach allows us to quantify the spatial uncertainty inherent in each label.
Spatial Distribution and Jaccard IoU (JIoU)
We define a new representation for probabilistic bounding boxes using a spatial probability density function (PDF), which describes the likelihood of any point in 3D space belonging to the object.
To evaluate detectors in this probabilistic space, we introduce Jaccard IoU (JIoU). JIoU extends the classic IoU metric by calculating the Jaccard index between the predicted spatial distribution and the ground truth distribution. Unlike standard IoU, JIoU penalizes over-confident predictions in regions of high label uncertainty, encouraging better-calibrated probabilistic outputs.
Key Insights: The "L-Shape" and Corner Localization
A critical observation from our generative model is the "L-shape" intuition: the corner of a vehicle closest to the LiDAR sensor is significantly more certain than its center or far corners.
Our analysis reveals that state-of-the-art detectors are often better at localizing the nearest corner than the object's center. By re-aligning predictions based on corners rather than centers, we observed a significant boost in AP across various architectures. For example, VoxelNet showed an $+8.0\%$ improvement in the "Hard" category on KITTI when using corner-alignment, suggesting that a large portion of localization error is actually "center-guessing" error.
Experimental Results
We validated our approach on the KITTI and Waymo Open Datasets. Key findings include:
- Superior Evaluation: JIoU provides a more nuanced assessment of probabilistic detectors by accounting for label quality.
- Label Noise Detection: The proposed model effectively identifies erroneous or highly ambiguous labels in existing datasets, outperforming traditional geometric heuristics.
- Architecture Sensitivity: Different detector architectures exhibit varying sensitivities to spatial uncertainty, with corner-based alignment providing consistent gains.
Conclusion and Future Work
This work provides a framework for inferring and utilizing spatial uncertainty in 3D object detection. By moving beyond deterministic labels to spatial distributions and JIoU, we enable more robust training and fairer evaluation of probabilistic models. Future research will focus on extending the generative model to non-rigid objects like pedestrians and integrating these uncertainty measures directly into detector loss functions to improve generalization in complex driving environments.
Summary updated