Calibrating the Uncertainty of Self-Supervised Monocular Depth without Labels

Jun Seok Kim, Rui HuangarXiv 2026 · arXiv

TL;DR

Self-supervised monocular depth networks are notoriously overconfident, often underestimating their own error by a factor of two or more. MonoCal is a new framework that fixes this miscalibration in both magnitude (scale) and distribution (shape) using only stereo pairs and model disagreement—no ground-truth labels required.

What Problem Does This Paper Address?

In autonomous driving and robotics, knowing how much to trust a depth estimate is just as important as the estimate itself. Self-supervised monocular depth networks can predict per-pixel uncertainty ($\sigma$), but there’s a catch: the training process doesn't actually constrain the magnitude of that uncertainty.

Standard evaluation metrics like AUSE (Area Under the Sparsification Error) only measure how well a network ranks pixels from most to least certain. If a network predicts an uncertainty that is consistently half the actual error, it still gets a perfect ranking score. In practice, however, downstream systems (like Kalman filters for SLAM) need the uncertainty to represent a physical quantity (e.g., standard deviation in meters).

The authors find that every major self-supervised network is miscalibrated. For instance, the popular MonoProb model has an absolute depth error nearly twice as large as its predicted uncertainty. This paper aims to fix both the scale (the average magnitude) and the shape (how uncertainty behaves across different depth ranges and confidence levels) without needing expensive LiDAR ground truth.

Methods

The authors introduce MonoCal, which consists of two distinct components:

  1. Scale Calibration (The "How Much" Fix): To find the correct scaling constant, the authors use stereo image pairs. By comparing the monocular network's depth map to a depth map generated via traditional semi-global matching (SGM) on the stereo pair, they can calculate a "stereo residual." The median ratio of this residual to the predicted $\sigma$ provides a calibration constant. Crucially, stereo pairs are only needed once to calculate this constant; inference remains purely monocular.

  2. Disagreement Target (The "Where" Fix): A simple constant can't fix "shape" errors—for example, if a model is more overconfident at close range than at long range. To fix this, the authors retrain the uncertainty head. Instead of trying to predict the unknown ground-truth error, the network is trained to predict the disagreement between its own depth estimate and that of a second, independently trained self-supervised network with a different architecture. If two different models disagree on a pixel's depth, it’s a strong signal that the area is uncertain.

Datasets

The primary experiments were conducted on the KITTI dataset (Eigen split). To test generalization, the authors also evaluated on:

  • Argoverse 1
  • DrivingStereo
  • nuScenes (used to test the limits of the method, as it lacks stereo pairs for calibration).

Key Insights and Ideas

  • The Calibration Gap: The paper reveals a massive gap in current research. While we've become good at ranking uncertainty, we've ignored its magnitude. The scale of miscalibration varies wildly across architectures (factors of 0.6 to 6).
  • Stereo as a Proxy: You don't need perfect ground truth to calibrate. Even though stereo matching has its own errors, the median residual across a few hundred images is stable enough to match a constant fitted with high-end LiDAR.
  • Architecture Disagreement is a Feature: Using the "disagreement" between two models as a training target is a clever way to bypass the lack of labels. It effectively uses the collective "ignorance" of different architectures to map out where errors are likely to occur.
  • Shape Matters: A constant scaling might fix the average error, but it often leaves the model overconfident at near distances and underconfident at far distances. The retraining step (Disagreement Target) is essential for a "flat" calibration curve across all depth bins.

Key Results

  • Scale Correction: On KITTI, MonoCal reduced the ratio of error to uncertainty ($r_{med}$) for MonoProb from 1.94 down to 0.96 (where 1.0 is perfectly calibrated).
  • Shape Correction: MonoCal significantly improved "Expected Normalized Calibration Error" (ENCE). It flattened the reliability curves, meaning the uncertainty became a reliable predictor of error regardless of whether the object was 5 meters or 50 meters away.
  • Outperforming Labels: Surprisingly, MonoCal’s label-free calibration often achieved better consistency across depth ranges than simply rescaling the model using ground-truth LiDAR data.
  • Qualitative Improvement: Visually, MonoCal produces "smoother" uncertainty maps that correlate better with actual structural errors (like object boundaries) rather than just following image textures like road markings.

Limitations and Future Work

  • Stereo Dependency: The scale calibration still requires stereo pairs during the calibration phase. For datasets like nuScenes that only provide monocular video, the authors had to "borrow" a constant from KITTI, which resulted in a less accurate scale (1.4 instead of ~1.0).
  • Retraining Requirement: While the scale calibration is "plug-and-play," fixing the shape requires retraining the uncertainty head of the network.
  • Future Directions: The authors suggest looking for ways to measure the calibration constant without a second sensor (like stereo or LiDAR) and exploring how these calibrated uncertainties improve downstream tasks like 3D reconstruction and SLAM.

Summary updated

Abstract

Self-supervised monocular depth networks can predict a per-pixel uncertainty, which is mainly evaluated by how well it ranks pixels according to their error. Such metrics are insensitive to the magnitude of the uncertainty, so an uncertainty half as large as the actual error obtains the same score. In this paper, we investigate whether the predicted uncertainty is as large as the depth error, both on average (scale) and across confident and uncertain, near and far pixels (shape). On KITTI, the scale is miscalibrated for every uncertainty estimate we evaluate, and the error of MonoProb is nearly twice its predicted uncertainty. We propose MonoCal, which corrects both the scale and the shape without ground-truth depth. Scale calibration multiplies the uncertainty by a single constant measured from stereo pairs, requiring no retraining, and this constant closely matches the one fitted with ground truth. Since a constant cannot change the shape, we further retrain the uncertainty to predict where the depth of the network disagrees with that of a second self-supervised network. Experiments on KITTI show that MonoCal achieves better calibration across uncertainty levels and depth ranges than rescaling with ground truth, while still using a single image at inference. Code will be made publicly available.
BibTeX
@misc{kim2026calibrating,
  author = {Jun Seok Kim and Rui Huang},
  title = {{Calibrating the Uncertainty of Self-Supervised Monocular Depth without Labels}},
  year = {2026},
  eprint = {2611.19188},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19188},
}
Figure