Inferring Spatial Uncertainty in Object Detection

Zining Wang, Di Feng, Yiyang Zhou, Wei Zhan, Lars Rosenbaum, Fabian Timm, Klaus Dietmayer, Masayoshi Tomizuka

Abstract

The availability of real-world datasets is the prerequisite for developing object detection methods for autonomous driving. While ambiguity exists in object labels due to error-prone annotation process or sensor observation noises, current object detection datasets only provide deterministic annotations without considering their uncertainty. This precludes an in-depth evaluation among different object detection methods, especially for those that explicitly model predictive probability. In this work, we propose a generative model to estimate bounding box label uncertainties from LiDAR point clouds, and define a new representation of the probabilistic bounding box through spatial distribution. Comprehensive experiments show that the proposed model represents uncertainties commonly seen in driving scenarios. Based on the spatial distribution, we further propose an extension of IoU, called the Jaccard IoU (JIoU), as a new evaluation metric that incorporates label uncertainty. Experiments on the KITTI and the Waymo Open Datasets show that JIoU is superior to IoU when evaluating probabilistic object detectors.

Introduction

In autonomous driving, training robust object detectors relies heavily on large-scale datasets like KITTI and Waymo. Traditionally, these datasets provide "ground truth" labels as deterministic bounding boxes. However, these labels are often inherently uncertain due to sensor noise and occlusions. For instance, LiDAR sensors typically only capture "L-shaped" reflections from the surfaces of a vehicle facing the sensor, leaving the far side of the object to be estimated by human annotators.

Standard evaluation metrics, such as Intersection over Union (IoU) and Average Precision (AP), ignore this ambiguity. This leads to two primary challenges: models are forced to overfit noisy labels during training, and probabilistic detectors—which output uncertainty estimates—cannot be accurately benchmarked against deterministic ground truths.

Methodology

Generative Label Uncertainty Model

To address these limitations, we propose a generative framework that treats bounding box parameters not as fixed values, but as random variables. Using Variational Bayes, we infer a posterior distribution of the bounding box parameters ($c_x, c_y, l, w, \theta$) conditioned on the observed LiDAR point cloud. This approach allows us to quantify the spatial uncertainty inherent in each label.

Inferred spatial probability distributions for vehicle bounding boxes based on LiDAR point clouds. The zoomed-in view illustrates how the model captures higher certainty at the observed corner.

Spatial Distribution and Jaccard IoU (JIoU)

We define a new representation for probabilistic bounding boxes using a spatial probability density function (PDF), which describes the likelihood of any point in 3D space belonging to the object.

To evaluate detectors in this probabilistic space, we introduce Jaccard IoU (JIoU). JIoU extends the classic IoU metric by calculating the Jaccard index between the predicted spatial distribution and the ground truth distribution. Unlike standard IoU, JIoU penalizes over-confident predictions in regions of high label uncertainty, encouraging better-calibrated probabilistic outputs.

Key Insights: The "L-Shape" and Corner Localization

A critical observation from our generative model is the "L-shape" intuition: the corner of a vehicle closest to the LiDAR sensor is significantly more certain than its center or far corners.

Spatial uncertainty distribution within a bounding box. The model identifies lower variance (σ) at the corners compared to the center, reflecting the nature of LiDAR observations.

Our analysis reveals that state-of-the-art detectors are often better at localizing the nearest corner than the object's center. By re-aligning predictions based on corners rather than centers, we observed a significant boost in AP across various architectures. For example, VoxelNet showed an $+8.0\%$ improvement in the "Hard" category on KITTI when using corner-alignment, suggesting that a large portion of localization error is actually "center-guessing" error.

Experimental Results

We validated our approach on the KITTI and Waymo Open Datasets. Key findings include:

  • Superior Evaluation: JIoU provides a more nuanced assessment of probabilistic detectors by accounting for label quality.
  • Label Noise Detection: The proposed model effectively identifies erroneous or highly ambiguous labels in existing datasets, outperforming traditional geometric heuristics.
  • Architecture Sensitivity: Different detector architectures exhibit varying sensitivities to spatial uncertainty, with corner-based alignment providing consistent gains.

Conclusion and Future Work

This work provides a framework for inferring and utilizing spatial uncertainty in 3D object detection. By moving beyond deterministic labels to spatial distributions and JIoU, we enable more robust training and fairer evaluation of probabilistic models. Future research will focus on extending the generative model to non-rigid objects like pedestrians and integrating these uncertainty measures directly into detector loss functions to improve generalization in complex driving environments.

Summary updated

Abstract

The availability of real-world datasets is the prerequisite for developing object detection methods for autonomous driving. While ambiguity exists in object labels due to error-prone annotation process or sensor observation noises, current object detection datasets only provide deterministic annotations without considering their uncertainty. This precludes an in-depth evaluation among different object detection methods, especially for those that explicitly model predictive probability. In this work, we propose a generative model to estimate bounding box label uncertainties from LiDAR point clouds, and define a new representation of the probabilistic bounding box through spatial distribution. Comprehensive experiments show that the proposed model represents uncertainties commonly seen in driving scenarios. Based on the spatial distribution, we further propose an extension of IoU, called the Jaccard IoU (JIoU), as a new evaluation metric that incorporates label uncertainty. Experiments on the KITTI and the Waymo Open Datasets show that JIoU is superior to IoU when evaluating probabilistic object detectors.
BibTeX
@inproceedings{wang2020inferring,
  author = {Zining Wang and Di Feng and Yiyang Zhou and Wei Zhan and Lars Rosenbaum and Fabian Timm and Klaus Dietmayer and Masayoshi Tomizuka},
  title = {{Inferring Spatial Uncertainty in Object Detection}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=gLC8mLY9Dkk},
}
Figure