DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution

Zhengxue Wang, Zhiqiang Yan, Jinshan Pan, Guangwei Gao, Kai Zhang, Jian Yang

Abstract

Depth Super-Resolution (DSR) is essential for enhancing low-resolution depth maps captured by consumer-grade sensors. However, real-world degradations are often unknown and complex, causing traditional methods—which assume simple bicubic downsampling—to fail. We present DORNet, a Degradation Oriented and Regularized Network for blind DSR. DORNet employs a self-supervised degradation learning strategy to implicitly model unknown degradations and a targeted RGB-D fusion mechanism to sharpen depth maps. Experiments demonstrate that DORNet significantly outperforms state-of-the-art methods on real-world datasets while maintaining high efficiency for practical applications.

Introduction

Depth maps captured by consumer-grade sensors, such as Time-of-Flight (ToF) cameras in smartphones, are frequently low-resolution and plagued by noise. While guided DSR aims to address this by using a high-resolution RGB image as a reference, most existing methods rely on the assumption that degradation is known and follows a simple bicubic downsampling process.

In real-world scenarios, this assumption is often invalid. Physical sensors introduce complex degradations, including structural distortion, blur, and noise influenced by lighting or surface reflectivity. When traditional models encounter these "blind" (unknown) degradations, their performance degrades significantly, resulting in blurry edges and misaligned structures. DORNet addresses this challenge by learning to model unknown degradations implicitly, without requiring explicit degradation labels.

Methodology

The DORNet framework consists of two primary branches designed to handle unknown degradations and optimize feature fusion.

Self-Supervised Degradation Learning

Instead of assuming a fixed degradation model, DORNet uses a Degradation Encoder to extract an implicit representation ($D$) directly from the low-resolution input. To regularize this learning, a Router dynamically selects the most appropriate degradation kernel generators (of varying sizes) to "re-degrade" the predicted high-resolution depth. By forcing this re-degraded version to match the original low-resolution input, the model learns underlying degradation patterns in a self-supervised loop. A contrastive loss further refines this by pushing the re-degraded depth closer to the low-resolution input and away from the clean ground truth.

Detailed architecture of DORNet, highlighting the self-supervised degradation learning and degradation-oriented fusion branches.

Degradation-Oriented Feature Transformation (DOFT)

Standard fusion methods often combine RGB and depth data indiscriminately. In contrast, the Degradation-Oriented Feature Transformation (DOFT) module uses learned degradation priors to selectively propagate information from the RGB image. It utilizes Deformable Convolutions (DCN) guided by the degradation representation to focus on RGB features that specifically sharpen edges or fill gaps where the depth signal is most degraded.

Experimental Results

DORNet was evaluated on three major benchmarks: the real-world RGB-D-D dataset, the TOFDSR dataset, and the synthetic NYU-v2 dataset.

Performance and Efficiency

On the real-world RGB-D-D dataset, DORNet achieved an RMSE of 3.42 cm, outperforming the previous state-of-the-art method, SFG (3.88 cm), and significantly surpassing SGNet (5.32 cm). Beyond accuracy, the model is highly efficient. A lightweight version, DORNet-T, achieves 161 FPS on an NVIDIA 4090 GPU, making it highly suitable for real-time AR/VR applications.

Visual comparison of real-world vs. synthetic degradation and qualitative results of DORNet against other methods.

Robustness to Real-World Noise

As shown in the visual comparisons and histograms, real-world degradation differs significantly from synthetic bicubic downsampling. DORNet's ability to model these complex patterns allows it to maintain a low error rate even under varying levels of Gaussian noise, where traditional models typically fail.

Conclusion

DORNet provides a robust and efficient solution for blind Depth Super-Resolution by moving away from fixed degradation assumptions. By learning implicit degradation representations and utilizing targeted RGB-D fusion, it bridges the gap between synthetic training and real-world application. Future work may extend this degradation-oriented strategy to other multi-modal tasks, such as satellite pan-sharpening or LiDAR depth completion.

Summary updated

Abstract

Recent RGB-guided depth super-resolution methods have achieved impressive performance under the assumption of fixed and known degradation (e.g., bicubic downsampling). However, in real-world scenarios, captured depth data often suffer from unconventional and unknown degradation due to sensor limitations and complex imaging environments (e.g., low reflective surfaces, varying illumination). Consequently, the performance of these methods significantly declines when real-world degradation deviate from their assumptions. In this paper, we propose the Degradation Oriented and Regularized Network (DORNet), a novel framework designed to adaptively address unknown degradation in real-world scenes through implicit degradation representations. Our approach begins with the development of a self-supervised degradation learning strategy, which models the degradation representations of low-resolution depth data using routing selection-based degradation regularization. To facilitate effective RGB-D fusion, we further introduce a degradation-oriented feature transformation module that selectively propagates RGB content into the depth data based on the learned degradation priors. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our DORNet in handling unknown degradations, outperforming existing methods.
BibTeX
@inproceedings{wang2025dornet,
  author = {Zhengxue Wang and Zhiqiang Yan and Jinshan Pan and Guangwei Gao and Kai Zhang and Jian Yang},
  title = {{DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.01474},
}
Figure