Monocular depth estimation (MDE) is often described as an "inherently ill-posed" problem. Mathematically, a single 2D image can correspond to an infinite number of possible 3D scene configurations. When a neural network looks at a photo, it doesn't have the luxury of stereo parallax or motion cues; it must infer distance from ambiguous visual signals like perspective, texture gradients, and object size.
Recent research has moved beyond simple pixel-wise regression to solve this ambiguity. By treating depth as a structured classification problem, incorporating geometric priors like "piece-wise planes," and using global attention to resolve local uncertainties, modern models are turning this "magic trick" into a precise science.
From Continuous Regression to Ordinal Discretization
Early attempts at MDE treated depth as a standard continuous regression task. However, training deep networks to predict high-resolution continuous maps is difficult to optimize. As Huan Fu and colleagues argued in A Compromise Principle in Deep Monocular Depth Estimation, the optimization landscape for raw regression is often too complex for networks to navigate effectively.
The breakthrough came with DORN (Deep Ordinal Regression Network), introduced by Huan Fu et al. in Deep Ordinal Regression Network for Monocular Depth Estimation. Instead of predicting an exact distance, DORN discretizes depth into intervals using a "spacing-increasing discretization" strategy. This recognizes that we need more precision for nearby objects than for distant ones. By reframing depth as an ordinal regression problem, the model can leverage the strengths of classification—which is easier to optimize—while maintaining the ordered nature of distance.
More recently, Fisher Yu pushed this further with iDisc, which uses an "Internal Discretization" module. This allows the network to learn its own discrete representation of a scene's geometry without being forced into a rigid, pre-defined grid.
Exploiting Geometric and Structural Priors
To resolve the ambiguity of a 2D projection, models must "know" something about how the physical world is built. One of the most powerful priors is the concept of coplanarity: the idea that many points in a scene (like walls, floors, or tabletops) lie on the same flat plane.
Lam Huynh and others introduced the Depth-Attention Volume (DAV), an attention mechanism that implicitly teaches the network to group pixels that belong to the same plane. Similarly, Shuwei Shao developed NDDepth, which treats scenes as a collection of "piece-wise planes." By predicting surface normals and plane distances alongside raw depth, the model ensures that the resulting 3D reconstruction is mathematically consistent and sharp.
Another approach involves integrating classical physics directly into the architecture. In VA-DepthNet, Ce Liu and Suryansh Kumar used a variational approach to predict depth gradients. This forces the network to respect geometric boundaries, preventing the "bleeding" effect where depth values spill over the edges of objects.
Global Context and the Transformer Shift
Local visual cues are often misleading. A pattern on a rug might look like a geometric edge, or a reflection on a window might look like a distant object. These are known as "visual pits." To overcome them, models need global context—they need to see the whole room to understand the floor.
The field has seen a significant shift toward Transformer-based architectures to capture these long-range dependencies. Shuwei Shao proposed TEDepth, which uses a Transformer-CNN ensemble to combine local precision with global scene understanding.
Perhaps the most intuitive use of attention is found in RED-T by Kyuhong Shim. This model uses its own predicted relative depth to guide its self-attention. By encouraging the network to attend to pixels at similar depths, it becomes robust against misleading textures, effectively "filtering out" patterns that don't align with the underlying 3D structure.
Semantic Divide and Conquer
Finally, researchers are leveraging semantic guidance to simplify the depth estimation task. If a model knows that a group of pixels represents a "car," it can apply a learned prior about how big cars usually are and how they sit on the ground.
In SDC-Depth, the scene is broken into semantic segments. The model predicts a "canonical" depth for each object type independently before stitching them back together. This "divide-and-conquer" strategy, also explored by Qing Mao in Semantic-Guided Representation Enhancement, ensures that object boundaries are preserved and that the model's depth guesses are grounded in the semantic reality of the scene.