Monocular Depth Estimation via Learned Geometric Priors

This direction is merged into another direction: monocular depth ambiguity (ill-posed 3D reconstruction). Its last version stays readable and citable here.

Version 1 of 1326 papers330 conceptsfrozen 2026-10-04

In computer vision, recovering the third dimension from a single 2D image is famously described as an "ill-posed" or "mathematically degenerate" problem. When a 3D scene is projected onto a 2D camera plane, the distance information for every pixel is lost. Theoretically, an infinite number of distinct 3D configurations could produce the exact same 2D image.

Despite this ambiguity, humans navigate the world with ease using only one eye. We do this by relying on "monocular cues"—contextual priors like the known size of objects, vanishing points, and texture gradients. Research in this field aims to give machines a similar ability to "hallucinate" depth by learning these priors from data.

From Planar Assumptions to Pixel Maps

Early attempts to solve this problem focused on structured environments. In Learning 3-D Scene Structure from a Single Still Image, Ashutosh Saxena and Andrew Y. Ng modeled the world as a collection of small, oriented planes (superpixels). By using a Markov Random Field (MRF), they could infer the orientation and depth of these planes relative to one another, effectively "popping up" a flat photo into a 3D model.

The field shifted significantly with the advent of deep learning, which allowed for dense, pixel-level predictions. David Eigen and Rob Fergus demonstrated in Predicting Images using Convolutional Networks that a single convolutional neural network (CNN) could be trained to output a "pixel map"—a 2D array where every pixel represents a depth value or a surface normal. This moved the field away from rigid geometric models toward a more flexible, data-driven understanding of scene layout.

Enforcing Geometric Consistency

While deep networks are excellent at recognizing patterns, they often struggle with global geometric consistency. A network might predict the depth of a chair correctly but fail to realize that the chair must be resting on the floor.

To fix this, researchers began integrating structural constraints directly into the learning process. Fayao Liu and Chunhua Shen introduced Deep Convolutional Neural Fields, which combined CNNs with Conditional Random Fields (CRFs) to ensure that neighboring pixels with similar appearances had similar depths.

Later, Wei Yin and Chunhua Shen proposed the "Virtual Normal" loss in Enforcing Geometric Constraints of Virtual Normal for Depth Prediction. By sampling triplets of points in the predicted 3D space and calculating the normal of the plane they form, the model learns to maintain long-range geometric relationships, leading to much sharper and more physically plausible reconstructions.

Learning Without Labels

One of the biggest hurdles in monocular depth estimation is the difficulty of obtaining ground-truth data. Measuring the exact distance of every pixel in a real-world scene usually requires expensive LiDAR sensors.

To bypass this, researchers developed self-supervised methods. The intuition is simple: if you have a stereo pair of images (left and right), a model can predict the depth of the left image and then use that depth to "warp" the left image to look like the right one. If the warped image matches the real right image, the depth prediction must be accurate.

Rares Andrei Ambrus and Adrien Gaidon pushed this further with SuperDepth, showing that high-resolution processing is critical for capturing fine-grained details. Other approaches, like T2Net by Chuanxia Zheng and Jianfei Cai, use synthetic data (from video games or simulations) and "translate" it to look like real-world photos, allowing models to learn from "free" labels.

Expanding the Scope

Modern research has expanded monocular estimation beyond simple static scenes:

  • Human Centricity: Models like ROMP by Michael J. Black and colleagues can recover the 3D mesh of multiple people from a single image, resolving complex occlusions where people appear to overlap.
  • Dynamic Scenes: In specialized fields like surgery, frameworks like BASED use Neural Radiance Fields (NeRFs) to reconstruct deforming tissues from monocular endoscopic video, overcoming the lack of fixed landmarks.
  • Efficiency: New architectures like PoseSSM by Hanxi Yin use State Space Models to "lift" 2D poses into 3D with much lower computational overhead than traditional Transformers.

By combining deep representational power with fundamental geometric laws, this research direction continues to narrow the gap between the 2D images we capture and the 3D world we inhabit.

Papers in this version

Showing 60 of 326 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_monocular_depth_estimation_via_learned_g_7cc0690a,
  title = {Monocular Depth Estimation via Learned Geometric Priors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/monocular-depth-estimation-via-learned-geometric-priors/rdr_7cc0690a461b4100b5d5fca8d34aebd0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_monocular_depth_estimation_via_learned_g_7cc0690a_v1,
  title = {Monocular Depth Estimation via Learned Geometric Priors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/monocular-depth-estimation-via-learned-geometric-priors/rdr_7cc0690a461b4100b5d5fca8d34aebd0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}