Reliable and Calibrated Semantic Occupancy Prediction by Hybrid Uncertainty Learning

Song Wang, Zhongdao Wang, Jiawei Yu, Wentong Li, Bailan Feng, Junbo Chen, Jianke ZhuIJCAI 2025 · DOI

Introduction

Semantic occupancy prediction is a critical task for autonomous driving, involving the partitioning of 3D space into a voxel grid where each cell is assigned a semantic label (e.g., vehicle, sidewalk, or empty). While vision-centric (camera-only) models have seen rapid progress in accuracy, narrowing the gap with expensive LiDAR-based systems, they often suffer from poor reliability. Specifically, these models tend to be "overconfident"—assigning high probability to incorrect labels. In safety-critical applications, it is paramount that a system can signal its own uncertainty. This work introduces RELIOCC, a framework designed to bridge this reliability gap through hybrid uncertainty learning and calibration.

Methodology: Hybrid Uncertainty Learning

RELIOCC is a "plug-and-play" framework that can be integrated into existing state-of-the-art occupancy networks like VoxFormer or SGN. Its core innovation lies in a dual-path uncertainty learning scheme that addresses both individual voxel confidence and inter-voxel relationships.

Overview of the RELIOCC framework, detailing the Absolute and Relative Uncertainty learning components.

Absolute and Relative Uncertainty

Most existing methods focus solely on Absolute Uncertainty, which estimates the model's confidence for an individual voxel in isolation. RELIOCC supplements this with Relative Uncertainty. By employing a "mix-up" strategy that blends features from different voxels, the model is forced to learn the relationships between pairs of voxels. This prevents the optimization process from stalling when absolute uncertainty values become small, leading to a more robust representation of "doubt."

Uncertainty-Aware Calibration

To ensure that predicted confidence scores align with actual accuracy, RELIOCC employs uncertainty-aware calibration. This can be performed online during training or offline as a post-processing step. In the offline mode, a learned "uncertainty-aware temperature" scales the model's output logits, ensuring that a confidence score (e.g., 0.9) corresponds to the true likelihood of being correct (90%).

Experimental Results

The framework was evaluated on the SemanticKITTI dataset. Beyond standard accuracy (mIoU), the authors utilized reliability-focused metrics: Prediction Rejection Ratio (PRR), which measures the ability to reject incorrect predictions, and Expected Calibration Error (ECE), which quantifies the gap between confidence and accuracy.

Quantitative Improvements

Integrating RELIOCC led to significant gains across multiple architectures. For the VoxFormer model, the ECE dropped from 5.90% to 2.84% in online mode, and further to 2.05% in offline mode. Simultaneously, the PRR for semantic labels improved from 42.97% to 47.75%.

Visual Honesty and Robustness

A key benefit of RELIOCC is its "visual honesty." As shown in the comparison below, baseline models often exhibit low uncertainty in regions where they are making significant errors. RELIOCC correctly identifies these error-prone regions with high uncertainty scores (highlighted in yellow).

Qualitative comparison of error maps and uncertainty maps between the SGN baseline and RELIOCC.

Furthermore, RELIOCC-enabled models demonstrate superior robustness to "out-of-domain" noise, such as sensor failures (dropped frames) or adverse weather conditions (rain, fog). In these scenarios, the model appropriately increases its reported uncertainty rather than producing confident but erroneous predictions.

Conclusion

RELIOCC addresses the critical need for reliability in vision-centric 3D semantic occupancy. By introducing relative uncertainty learning and specialized calibration, it enables models to "know what they don't know." This transparency is essential for downstream autonomous driving tasks, such as path planning and decision-making, where a reliable uncertainty estimate can prevent catastrophic failures.

Summary updated

BibTeX
@inproceedings{wang2025reliable,
  author = {Song Wang and Zhongdao Wang and Jiawei Yu and Wentong Li and Bailan Feng and Junbo Chen and Jianke Zhu},
  title = {{Reliable and Calibrated Semantic Occupancy Prediction by Hybrid Uncertainty Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/220},
}
Figure