When a three-dimensional scene is projected onto a two-dimensional image plane, a fundamental piece of information is lost: the absolute physical scale. If we multiply all spatial coordinates and distances in a scene by a constant factor $\alpha$, its projection onto the camera sensor remains completely unchanged. Because of this inherent scale ambiguity, standard monocular vision systems are "scale-blind"—unable to distinguish a miniature toy model from a full-sized object without external information.
To transform relative depth maps into absolute physical measurements (meters, centimeters, or square areas), computer vision systems must anchor the visual data to a known physical reference. Recent research tackles this problem by leveraging geometric, semantic, optical, and data-driven constraints to recover the missing metric scale directly from monocular input.
Geometric Anchors: Camera Height and the Ground Plane
The most direct way to resolve scale ambiguity is to anchor the scene geometry to a known physical dimension of the camera setup itself. In many practical applications, such as autonomous driving or robotics, the camera is mounted at a fixed, known height $h$ above a flat ground plane.
This simple physical constraint completely changes the mathematics of depth estimation. If a vision system can identify which pixels belong to the ground plane, it can intersect the camera rays corresponding to those pixels with a theoretical plane equation defined by $h$. Through basic trigonometry, the absolute distance to every ground pixel is instantly resolved.
Camera (Known Height: h)
o
| \
| \ Camera Ray
| \
---|------X----------- Ground Plane
<------->
Absolute Distance Calculated via Trigonometry
Aurélien Cecille et al. leverage this mechanism in GroCo: Ground Constraint for Metric Self-supervised Monocular Depth. By enforcing geometric consistency between predicted depth maps and a theoretical ground plane derived from the camera's mounting height during self-supervised training, the model propagates absolute metric units across the entire 3D scene.
Similarly, Jinchang Zhang et al. demonstrate in Embodiment: Self-Supervised Depth Estimation Based on Camera Models that treating the camera as an embodied agent with fixed physical parameters transforms depth estimation from an unconstrained prediction task into a bounded trigonometric calculation.
Once the scale of the ground plane is metrically resolved, it acts as a spatial scaffold. As René Ranftl et al. show in Dense Monocular Depth Estimation in Complex Dynamic Scenes, enforcing physical connectivity constraints—such as the requirement that dynamic objects must be supported by the ground—allows the system to mathematically determine the absolute scale and depth of moving objects in the environment.
Sadra Safadoust further demonstrates in DepthP+P: Metric Accurate Monocular Depth Estimation using Planar and Parallax that integrating this known camera height into a classical planar parallax framework allows self-supervised models to achieve metric accuracy without requiring post-hoc scaling adjustments against ground-truth depth maps.
Semantic Anchors: Leveraging Object Dimensions
Humans do not just use geometry to perceive scale; we use our semantic understanding of the world. We know that an average door frame is roughly 2 meters tall, and an average sedan is about 4.5 meters long. By identifying known object categories, a vision system can use these size priors as a metric anchor.
If the physical size of an object category is known, its projected size on the 2D image plane becomes a direct function of its distance from the lens. Marius Beul et al. demonstrate this in Visually Guided Balloon Popping with an Autonomous MAV at MBZIRC 2020, showing that comparing a target's known physical dimensions to its 2D image radius provides highly reliable 3D localization at long ranges, bypassing the high noise levels typical of stereo depth sensors in outdoor environments.
With the advent of large language models (LLMs) and extensive semantic databases, these size priors can be generalized to almost any object. Georgy Ponimatkin et al. exploit this in 6D Object Pose Tracking in Internet Videos for Robotic Manipulation by querying an LLM for the typical physical dimensions of a detected object category. This semantic knowledge provides a metric scale prior that transforms relative depth trajectories into absolute 3D coordinates suitable for precise robotic control.
2D Image Plane Semantic Prior (LLM / Database)
+---------------+ +----------------------------+
| [Object] | ----> | "Average Sedan = 4.5m long"|
+---------------+ +----------------------------+
| |
v v
Projected Size ------------> Absolute 3D Distance
This concept can also be scaled up to entire environmental layouts. In JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenes, Haimei Zhao et al. resolve scale ambiguity by mathematically linking the camera's perspective view with a bird's-eye-view layout that uses a fixed metric grid. Projecting this top-down metric distance field back into the camera's perspective establishes a scale anchor that forces the neural network to learn absolute physical distances.
Optical and Structural Distortions
Scale information can also be recovered by exploiting the physical properties of the camera optics or non-standard projection geometries, even when semantic context or camera height is unavailable.
In standard perspective cameras, light rays converge at a single focal point, which causes the scale-blindness problem. However, non-central projection systems break this rule. Josechu Guerrero demonstrates in Scaled 360 Layouts: Revisiting Non-Central Panoramas that non-central panoramas—where light rays do not intersect at a single point—inherently encode absolute scale information within a single image. The geometric distortions caused by the non-central rays create a virtual internal baseline, allowing the system to triangulate distances and room dimensions without any prior knowledge of camera height.
Alternatively, the internal optical adjustments of a standard lens can serve as a metric sensor. Georges Younes et al. show in Scaled Monocular Visual SLAM that depth-from-focus (DfF) techniques can recover absolute metric scale during system initialization. By treating the lens's focus properties and internal optical metadata as primary depth measurements, the system bypasses the need for external sensors or manual distance inputs.
Implicit Scale from Geometric Foundation Models
Rather than relying on explicit geometric equations or manual semantic lookups for every frame, modern deep learning architectures can learn to infer metric scale implicitly. By training on massive, diverse datasets containing millions of images paired with ground-truth metric data (such as LiDAR), geometric foundation models learn the subtle structural and perspective cues that correlate with absolute distance.
A major challenge in this domain is focal length ambiguity: a tight crop from a long focal length lens can look identical to a close-up shot from a wide-angle lens, confusing the network's sense of scale. Chi Zhang et al. address this in Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation by introducing a canonical camera transformation. This method normalizes varying camera intrinsics into a standardized virtual camera space, allowing the model to deliver highly accurate, zero-shot metric depth predictions across entirely unseen environments.
Varying Images Canonical Transformation Zero-Shot Prediction
+---------------+ +-----------------------+ +--------------------+
| Wide-Angle | ----> | Transform to Standard | ----> | Accurate Metric |
| Telephoto | | Virtual Camera Space | | Depth & Normals |
+---------------+ +-----------------------+ +--------------------+
These pre-trained metric foundation models can then be integrated directly into downstream pipelines to solve classical vision problems. For instance, Junda Cheng et al. demonstrate in RoMeO: Robust Metric Visual Odometry that injecting a pre-trained monocular depth model into the initialization phase of a visual odometry pipeline provides an immediate estimate of absolute physical scale. This allows a single-camera system to track its trajectory and map an environment metrically over time without experiencing the scale drift or initialization failures that plague traditional, scale-blind monocular SLAM frameworks.