Degradation Oriented and Regularized Network for Real-World Depth Super-Resolution

Zhengxue Wang, Zhiqiang Yan, Jinshan Pan, Guangwei Gao, Kai Zhang, Jian Yang

Abstract

Recent RGB-guided depth super-resolution methods have achieved impressive performance under the assumption of fixed and known degradation (e.g., bicubic downsampling). However, in real-world scenarios, captured depth data often suffer from unconventional and unknown degradation due to sensor limitations and complex imaging environments (e.g., low reflective surfaces, varying illumination). Consequently, the performance of these methods significantly declines when real-world degradation deviate from their assumptions. In this paper, we propose the Degradation Oriented and Regularized Network (DORNet), a novel framework designed to adaptively address unknown degradation in real-world scenes through implicit degradation representations. Our approach begins with the development of a self-supervised degradation learning strategy, which models the degradation representations of low-resolution depth data using routing selection-based degradation regularization. To facilitate effective RGB-D fusion, we further introduce a degradation-oriented feature transformation module that selectively propagates RGB content into the depth data based on the learned degradation priors. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our DORNet in handling unknown degradation, outperforming existing methods.

Introduction

Depth sensors, such as Time-of-Flight (ToF) and LiDAR, are critical for applications in augmented reality, robotics, and autonomous driving. Despite their utility, these sensors often produce low-resolution (LR) depth maps characterized by complex, unknown degradations like structural noise and blur, particularly in environments with low reflectivity or challenging lighting.

Traditional Depth Super-Resolution (DSR) methods typically assume a simple, fixed degradation model, such as bicubic downsampling. This mismatch between synthetic assumptions and real-world sensor characteristics leads to significant performance degradation in practical applications. The fundamental challenge lies in training a model to restore depth maps when the specific degradation process is unknown and lacks explicit labels for supervision.

Methodology

DORNet addresses this challenge by explicitly modeling the degradation process through three integrated components:

  1. Self-Supervised Degradation Learning (DL): The network encodes the input LR depth into an implicit degradation representation, capturing the unique "fingerprint" of noise and blur for each specific image.
  2. Routing Selection-based Degradation Regularization (DR): Utilizing a Mixture-of-Experts (MoE) inspired approach, a Router dynamically selects kernel generators based on the learned degradation. These kernels "re-degrade" the predicted high-resolution (HR) depth back to the LR space. By minimizing the difference between this re-degraded output and the original input, the model learns underlying degradation patterns without human-provided labels.
  3. Degradation-Oriented Feature Transformation (DOFT): Unlike standard fusion techniques, DOFT uses learned degradation priors to selectively filter RGB information. It employs Deformable Convolutions (DCN) to align RGB guidance with areas requiring the most restoration, ensuring that high-frequency details are injected only where the depth data is unreliable.
The DORNet architecture featuring self-supervised degradation learning and degradation-oriented fusion modules.

Experimental Results

The performance of DORNet was evaluated on real-world datasets (RGB-D-D, TOFDSR) and synthetic benchmarks (NYU-v2). On the RGB-D-D dataset, captured with a Huawei P30 Pro ToF sensor, DORNet achieved a state-of-the-art RMSE of 3.42cm, outperforming the previous best method (SFG) by a significant margin.

Qualitative analysis demonstrates DORNet's ability to recover fine structures, such as thin chair legs, which are often lost in other methods. Furthermore, the "Tiny" version, DORNet-T, achieves high efficiency with only 0.46M parameters, processing at 161 FPS while still outperforming much larger models like SGNet.

Qualitative comparison of DSR methods and histogram analysis showing the distribution difference between real-world and synthetic LR depth.

Conclusion

DORNet introduces a robust framework for blind depth super-resolution by treating degradation modeling as a self-supervised task. By mimicking the degradation process and using it to guide feature fusion, the model achieves superior results on real-world sensor data. Future work includes addressing dependencies on RGB quality and extending the framework to video sequences for improved temporal consistency.

Summary updated

Abstract

Recent RGB-guided depth super-resolution methods have achieved impressive performance under the assumption of fixed and known degradation (e.g., bicubic downsampling). However, in real-world scenarios, captured depth data often suffer from unconventional and unknown degradation due to sensor limitations and complex imaging environments (e.g., low reflective surfaces, varying illumination). Consequently, the performance of these methods significantly declines when real-world degradation deviate from their assumptions. In this paper, we propose the Degradation Oriented and Regularized Network (DORNet), a novel framework designed to adaptively address unknown degradation in real-world scenes through implicit degradation representations. Our approach begins with the development of a self-supervised degradation learning strategy, which models the degradation representations of low-resolution depth data using routing selection-based degradation regularization. To facilitate effective RGB-D fusion, we further introduce a degradation-oriented feature transformation module that selectively propagates RGB content into the depth data based on the learned degradation priors. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our DORNet in handling unknown degradation, outperforming existing methods. The code is available at https://github.com/yanzq95/DORNet.
BibTeX
@inproceedings{wang2025degradation,
  author = {Zhengxue Wang and Zhiqiang Yan and Jinshan Pan and Guangwei Gao and Kai Zhang and Jian Yang},
  title = {{Degradation Oriented and Regularized Network for Real-World Depth Super-Resolution}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.01474},
}
Figure