Imagine looking at a photograph of a single object, like a chair. If the chair appears small in the frame, your brain faces an ambiguity: is it a normal-sized chair far away, a miniature toy chair close up, or did the photographer simply use a wide-angle lens?
For deep neural networks trained on monocular images, this puzzle is notoriously difficult. Standard monocular depth estimation models easily confuse the physical distance of an object with the optical properties of the camera that captured it. When a model trained on a specific camera setup is deployed on a new camera with a different focal length or resolution, it suffers from a phenomenon known as depth shift—severe localization and structural errors caused entirely by the change in hardware.
To build models that work universally across any camera, researchers have developed a suite of techniques to decouple scene geometry from camera intrinsics.
The Core Ambiguity: Focal Length and Scale
The root of the problem lies in perspective projection. A 3D point $(X, Y, Z)$ is mapped to 2D pixel coordinates $(u, v)$ via the camera's focal length $f$:
$$u = f \frac{X}{Z} + c_u$$
When a neural network attempts to regress the absolute metric depth $Z$ directly from visual features, it implicitly relies on the relationship between an object's pixel size $u$ and its typical physical size $X$. However, as Wang et al. observe, conventional depth estimation entangles the predicted depth with the specific focal length $f$ of the training camera. If the focal length changes, the pixel footprint of the object changes, tricking the model into miscalculating $Z$.
In fact, Zheng et al. demonstrated that variations in camera focal lengths are the primary cause of performance degradation when transferring 3D models across different sensor suites. To achieve true camera agnosticism, models must separate what is in the scene from how the camera is looking at it.
Decomposing Depth: Structure vs. Scale
One powerful strategy to bypass this ambiguity is to change the learning objective entirely. Instead of forcing a network to predict absolute metric distances, the task can be broken down into two distinct sub-tasks: predicting a scale-agnostic relative depth map, and predicting a global scale factor.
Raw Image ───► [ Neural Network ] ───► Relative Depth Map (Shapes & Boundaries)
│
▼
Absolute Metric Depth ◄─── [ Multiply ] ◄─── Global Scale Factor (Sensor Calibration)
Yan et al. show that this decomposition significantly simplifies network optimization. By isolating the geometric structure from scale ambiguity, the network can focus purely on spatial arrangements and object boundaries without being penalized by fluctuating global scales. Similarly, Jun et al. decompose metric depth maps into a scale-invariant normalized depth map and global statistical parameters like mean depth and standard deviation.
This formulation leads to what Yin et al. and Obukhov et al. define as an affine-invariant depth representation. By training models with losses that discount global scale and shift adjustments, the network learns the fundamental topological shape of environments. The resulting representation is inherently robust to unknown camera optics because the relative ordering of surfaces remains constant regardless of whether a scene is viewed through a telephoto or a wide-angle lens.
Intrinsics Normalization: Baking the Camera into the Target
If an application requires absolute metric depth rather than relative structure, the network must be made aware of the camera parameters, but in a way that normalizes the learning space.
A straightforward approach is to incorporate the camera's focal length directly into the optimization target. Moon et al. introduced this concept by normalizing distance estimation using camera intrinsics, ensuring that the relationship between image-space dimensions and physical distance remains consistent across different sensors.
Instead of expecting the network to output raw distance $Z$, the network can be trained to output a normalized depth target. For instance, Jia utilizes a normalized depth optimization target to ensure that objects appearing at the same scale in images with different intrinsic parameters map to consistent spatial coordinates.
Alternatively, the normalization can happen by transforming the data into a standardized virtual space. Zhang et al. implement a canonical camera transformation that standardizes the focal length during training. By adjusting the input images to simulate a fixed, universal camera geometry, the model is insulated from sensor-specific variations, allowing it to generalize zero-shot across diverse hardware.
Coordinate Shifts: Moving to Pseudo-Spherical Space
While normalizing focal lengths helps, standard depth estimation still suffers from geometric distortions because it operates in Cartesian coordinates ($X, Y, Z$). In a Cartesian framework, flat planes parallel to the camera sensor have varying distances from the optical center depending on how far a pixel is from the principal point. This couples the camera's field-of-view directly with the depth values.
To break this mathematical coupling, Piccinelli et al. and Sakaridis et al. proposed shifting the output representation from Cartesian space to pseudo-spherical coordinates.
Cartesian Representation (Z-Depth) Pseudo-Spherical Representation
┌───────────────┐ ▲
│ Flat Wall │ .───'│'───.
└───────┬───────┘ / │ \
│ │ Constant │
▼ │ Distance │
Camera Sensor \ │ /
(Edges are physically further '───.│.───'
from the center point) Camera Sensor
In a pseudo-spherical coordinate system, distance is measured along rays radiating from the optical center rather than perpendicular to the image plane. This formulation mathematically disentangles camera geometry from scene scale. Because the angular field-of-view is handled independently of the radial distance, errors or changes in camera intrinsics cannot propagate into the depth predictions, yielding highly stable zero-shot generalization.
Resolution and Pixel-Relative Adaptation
Camera agnosticism also requires handling variations in image resolution. If an image is downsampled or cropped, the pixel density changes, which can easily confuse a model into interpreting a change in resolution as a change in physical distance.
To decouple geometric distance from pixel-level density, Hui et al. introduced a cross-scale consistency loss. By penalizing discrepancies between depth maps generated from different scaled versions of the same input image, the model learns to ignore resolution artifacts and focus on scale-invariant structural cues.
Ultimately, by shifting the network's focus from absolute pixel values to normalized, affine-invariant, or pixel-relative representations—as explored by Chen et al.—modern depth estimators can successfully isolate pure geometry. The result is a robust, camera-agnostic understanding of 3D space that functions seamlessly across different devices, resolutions, and environments.
Key Foundational Work
CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth (2019) — contributed 3 concept elements to this direction.
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image (2019) — contributed 2 concept elements to this direction.
Perspective-n-Learned-Point: Pose Estimation from Relative Depth (2019) — contributed 2 concept elements to this direction.