Introduction
Monocular Depth Estimation (MDE)—the task of predicting 3D scene depth from a single 2D image—is a cornerstone of robotics, autonomous driving, and augmented reality. While recent "foundation models" like DepthAnything have achieved remarkable results by training on massive geometric datasets, they often struggle with semantic understanding. These models frequently produce blurry or over-smoothed depth maps when encountering complex, thin structures like power lines or tree branches because they lack the context of what an object actually is.
Training a new foundation model from scratch to incorporate semantic data is computationally prohibitive. This paper introduces BriGeS (Bridging Geometric and Semantic), a novel framework that injects semantic awareness into existing depth models efficiently and effectively.
The BriGeS Architecture
The core innovation of BriGeS is the integration of a frozen geometric foundation model (DepthAnything) with a frozen semantic foundation model (SegmentAnything, or SAM). Instead of retraining these massive networks, the authors introduce a lightweight Bridging Gate to fuse their features.
The Bridging Gate
The Bridging Gate extracts features from both encoders and uses a cross-attention mechanism. In this setup, the depth features act as the query, while semantic features act as the key and value. This allows the model to dynamically refine geometric predictions using semantic boundaries. To maintain efficiency, only the Bridging Gate is trained, while the massive encoders and the depth decoder remain frozen.
Attention Temperature Scaling
A key discovery during development was that cross-modal fusion can cause the attention mechanism to hyper-focus on the centers of objects, neglecting edges. To counteract this, the authors implemented Attention Temperature Scaling. By dividing attention scores by a temperature factor ($\tau = 2.5$) during inference, the model "cools down" its attention, spreading it more evenly across the entire object and its fine-grained boundaries.
Training and Evaluation
By focusing only on the Bridging Gate, BriGeS is remarkably parameter-efficient. It was trained on just 1% of the original DepthAnything training set (~600K samples) in roughly 8–18 hours on a standard 8-GPU setup.
Evaluation was conducted across several zero-shot benchmarks, including KITTI, NYUv2, ETH3D, and DIODE. The model was also tested on the DA-2K Benchmark, which specifically targets high-resolution scenes with intricate details.
Key Results
BriGeS consistently outperforms state-of-the-art models like MiDaS, DPT, and Marigold. Compared to the base DepthAnything V1 and V2 models, BriGeS achieved an average 7.33% reduction in Absolute Relative Error (AbsRel). On the challenging DIODE dataset, the improvement was even more pronounced, with a 15.33% reduction in error.
Visually, the difference is striking. As shown in the qualitative results, BriGeS prevents depth "bleeding" across edges. It successfully maps intricate fishing nets, thin branches, and power lines that diffusion-based or purely geometric models often blur or ignore.
Conclusion and Future Work
BriGeS demonstrates that semantic awareness is the "missing link" for high-fidelity monocular depth estimation. While the current dual-model approach is computationally heavy during inference, it proves that foundation models can be successfully bridged with minimal retraining. Future research will focus on knowledge distillation to compress this dual-model intelligence into a single, unified, and memory-efficient encoder.
Summary updated