Deep learning has revolutionized monocular and stereo depth estimation, yet these models often suffer from a "domain gap"—a sharp decline in performance when moved from specific training environments (like the KITTI driving dataset) to novel scenes (like indoor rooms or snowy weather). As noted by Sun et al., this lack of zero-shot robustness prevents depth models from being reliably deployed in unpredictable real-world settings.
The research direction focuses on bridging this gap by moving away from scene-specific fine-tuning toward universal models that understand the underlying geometry of the world rather than just the statistics of a single dataset.
Scaling Data and Foundation Models
One of the most effective ways to achieve generalization is through sheer scale. By training on massive, diverse datasets, models can learn features that are invariant to specific environments. Yang et al. introduced Depth Anything, a foundation model trained on 62 million unlabeled images. The authors argue that by leveraging strong semantic priors from Vision Foundation Models like DINOv2, the network can focus on high-level scene structure rather than low-level texture that varies between domains.
Similarly, in the stereo matching domain, Wen et al. proposed FoundationStereo, which uses a million-pair synthetic dataset and "side-tuning" from monocular models to achieve zero-shot generalization. The intuition here is that synthetic data provides perfect ground truth for geometry, while monocular priors provide the robustness needed to handle real-world visual complexity.
Test-Time Adaptation and Meta-Learning
Rather than building a single static model, some researchers propose models that adapt as they "see" new environments. Tonioni et al. pioneered a meta-learning framework that trains a model specifically for its ability to quickly fine-tune itself in an unsupervised manner when it encounters new scenes.
More recently, Sung et al. developed PITTA, a pose-agnostic and instance-aware framework for test-time adaptation. This approach allows a model to adjust to challenging conditions like fog or rain in real-time. By treating each new environment as a small learning problem, these methods mitigate the brittleness of models trained on static datasets.
Preserving Geometric and Structural Priors
A significant cause of generalization failure is the loss of metric scale and geometric consistency. Standard neural network layers, like Batch Normalization, are often designed to be scale-invariant, which is counterproductive for depth tasks. To solve this, Zheng et al. introduced Scale Propagation Normalization (SP-Norm), a layer that preserves absolute scale while maintaining training stability. This allows models to generalize across disparate environments, such as moving from outdoor driving to indoor rooms, without losing their sense of physical dimension.
Other approaches focus on enhancing the model's awareness of object boundaries and textures. Manzanera et al. used a distance transform over semantic contours to provide a clear geometric signal in featureless regions like flat walls, which are notoriously difficult for self-supervised models to handle.
Hybrid Architectures and Multi-Expert Systems
The choice of architecture also plays a crucial role in generalization. Bae argues that "shape bias" is the secret to real-world performance. By combining the local precision of CNNs with the global shape-awareness of Transformers in the MonoFormer architecture, the authors achieved significantly better generalization to unseen environments.
Furthermore, depth estimation models often struggle with the "long-tail" problem, where they perform well on common depth ranges but fail on rare ones. Yu et al. addressed this with DME (Distance-aware Multi-Expert), which uses specialized neural "experts" for different distance brackets. A pixel-level router then stitches these predictions together, ensuring that the model remains accurate whether it is looking at a close-up object or a distant horizon.