Error Propagation in Monocular Depth Priors

1,145 concepts812 papers8 proposals

Lifting a single 2D image into 3D space is a fundamentally ill-posed problem. To resolve this ambiguity without relying on expensive active sensors like LiDAR or stereo-vision systems, modern computer vision frameworks increasingly employ pre-trained monocular depth estimation models as a geometric anchor. Whether optimizing neural implicit surfaces, synthesizing novel views, or detecting objects in 3D, these monocular depth maps serve as a foundational spatial prior.

However, this paradigm introduces a critical vulnerability: the downstream system becomes entirely dependent on the fidelity of the initial depth prediction. When the monocular model misinterprets a scene, these errors propagate unchecked through subsequent optimization, fusion, or reasoning stages. The accuracy of the final 3D output is strictly bounded by the quality of the initial monocular estimate, creating a pervasive performance bottleneck across the field.

Geometric Distortions in Neural Rendering and Reconstruction

In neural implicit surface reconstruction and novel view synthesis, monocular depth maps are frequently used to guide point sampling and regularize geometry, especially in textureless or few-shot scenarios. However, when the underlying depth model fails, the neural rendering process inherits these flaws, resulting in severe geometric artifacts.

For instance, Zehao Yu et al. demonstrate in MonoSDF that errors or biases in external depth estimators propagate directly into the final signed distance functions (SDFs), distorting the reconstructed surface geometry. This issue is particularly acute when dealing with complex materials or sparse data. Jaeyoung Chung notes in Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images that when pre-trained monocular models fail to accurately interpret challenging materials like shiny or transparent surfaces, the resulting 3D Gaussian Splatting geometry directly inherits these inaccuracies.

Similarly, Malte Prinzler et al. show in DINER: Depth-aware Image-based NEural Radiance fields that noisy depth priors cause the subsequent neural rendering process to fail to produce a coherent 3D structure. The structural integrity of the scene collapses if the initial depth map lacks fine-grained details or introduces edge noise. As the authors of SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Images observe, a lack of detail or incorrect spatial relationships in the underlying depth map can lead to a total collapse of the perceived 3D effect.

This sensitivity is echoed by Raja Kumar et al. in Few-Shot Novel View Synthesis Using Depth Aware 3D Gaussian Splatting and Fangzhou Mu et al. in 3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image, where depth inaccuracies near intricate boundaries directly translate into structural artifacts in the synthesized views. Furthermore, Florian Kluger et al. observe in Cuboids Revisited: Learning Robust 3D Shape Fitting to Single RGB Images that systems deriving 3D abstractions from single RGB images suffer from error propagation where inaccuracies in the initial monocular depth estimation directly degrade the quality of the final geometric fitting.

Trajectory and Scale Failures in Geometric Optimization

The propagation of depth errors extends beyond static scene reconstruction into dynamic tracking, Structure-from-Motion (SfM), and visual odometry. In these applications, monocular depth is often leveraged to stabilize camera tracking or recover absolute metric scale.

However, integrating these priors makes the global optimization pipeline highly vulnerable to catastrophic failures in the underlying depth estimator. Sheng Liu 0017 et al. demonstrate in Depth-Guided Sparse Structure-from-Motion for Movies and TV Shows that errors caused by extreme lighting or unusual scene geometry in the initial depth maps propagate through the optimization process, leading to inaccurate camera trajectories or distorted 3D points. This vulnerability is a core constraint in hybrid geometric-neural systems; as Zador Pataki et al. argue in MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion, non-Lambertian surfaces like mirrors can propagate errors that disrupt the global structure-from-motion pipeline. This dependency is also a limiting factor when optimizing neural radiance fields without pose priors, as noted by Wenjing Bian et al. in NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior.

When these systems are deployed for metric scale recovery or robotic control, depth errors can lead to physically impossible outcomes. Junda Cheng et al. show in RoMeO: Robust Metric Visual Odometry that the reliability of metric scale recovery is strictly bounded by the generalization limits of the underlying depth models, where hallucinations degrade the precision of the final trajectory. In robotic manipulation tasks, such as the framework proposed by Hanzhi Chen et al. in VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation, inaccurate depth maps result in the generation of physically impossible actions that fail when mapped to real-world metric space.

The Bottleneck in Spatial Reasoning and Object Detection

For higher-level semantic tasks like 3D object detection and visual relationship prediction, monocular depth maps provide the necessary 3D spatial context to reason about object interactions. Yet, because these models operate downstream of the depth estimator, they inherit a strict performance bottleneck.

In Multi-Level Fusion Based 3D Object Detection From Monocular Images, Bin Xu et al. establish that errors in the initial depth map propagate through subsequent multi-level fusion stages, creating a bottleneck that can only be resolved with higher-fidelity spatial priors. Even when systems attempt to refine object positions in bird's-eye view (BEV) space, they remain constrained by the initial input. Yingjie CAI et al. show in Monocular 3D Object Detection with Decoupled Structured Polygon Estimation and Height-Guided Depth Estimation that if the initial depth estimation is inaccurate, subsequent spatial refinement in BEV space fails to correct the object's position effectively. This sensitivity creates a persistent performance gap between monocular approaches and active sensing technologies like LiDAR, as noted by the authors of GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving.

This error propagation similarly degrades relational and semantic reasoning. An-Chieh Cheng et al. demonstrate in Visual Relationship Prediction via Label Clustering and Incorporation of Depth Information that monocular estimation errors propagate through the prediction pipeline, leading to incorrect spatial relationship classifications. Whether predicting human-object interactions or tracking gaze targets, the downstream spatial reasoning is inherently constrained by the precision of the underlying depth estimator, a vulnerability highlighted by the authors of Multimodal Across Domains Gaze Target Detection and by Anoop Cherian et al. in (2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering.

Furthermore, Sheng Qian notes in Saliency Detection Integrated with Depth Information that integrating spatial information into visual processing creates a heavy dependency on depth accuracy; if the underlying depth map is noisy, downstream object identification or saliency detection will likely fail regardless of the quality of the visual features.

The Limits of Downstream Refinement

A natural response to this dependency is to introduce downstream refinement layers or spatial propagation networks to correct the initial depth maps. However, recent insights reveal that refinement frameworks are fundamentally limited by the structural completeness of the base monocular model.

Jinyoung Jun et al. explore this limitation in Masked Spatial Propagation Network for Sparsity-Adaptive Depth Refinement. They demonstrate that if the initial monocular estimator suffers from large-scale structural errors—such as completely missing an entire object—and no sparse sensor data is available in that region to provide a hard correction, the subsequent propagation and refinement process is entirely unable to recover the missing geometry.

This underscores the core challenge of the monocular depth dependency: downstream optimization and refinement can smooth out local noise, but they cannot easily correct catastrophic structural omissions or hallucinations propagated from the initial depth prior.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 748 papers
  2. Version 1 · 2026-10-04 · 643 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_depth_estimation_dependency_df452cc6,
  title = {depth estimation dependency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/depth-estimation-dependency/rdr_df452cc6fda94d75bde4da606b91188e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_depth_estimation_dependency_df452cc6_v2,
  title = {depth estimation dependency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/depth-estimation-dependency/rdr_df452cc6fda94d75bde4da606b91188e/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake