RGB Guided Depth Map Super-Resolution with Coupled U-Net

Yingjie Cui, Qingmin Liao, Wenming Yang, Jing-Hao Xue

Abstract

The depth maps captured by RGB-D cameras usually are of low resolution, entailing recent efforts to develop depth super-resolution (DSR) methods. However, several problems remain in existing DSR methods. First, conventional DSR methods often suffer from unexpected artifacts. Secondly, high-resolution (HR) RGB features and low-resolution (LR) depth features are often fused in shallow layers only. Thirdly, only the last layer of features is used for reconstruction. To address the above problems, we propose Coupled U-Net (CU-Net), a new color image guided DSR method built on two U-Net branches for HR color images and LR depth maps, respectively. The CU-Net embeds a dual skip connection structure to leverage the feature interaction of the two branches, and a multi-scale fusion to fuse the deeper and multi-scale features of two branch decoders for more effective feature reconstruction. Moreover, a channel attention module is proposed to eliminate artifacts. Extensive experiments show that the proposed CU-Net outperforms state-of-the-art methods.
BibTeX
@inproceedings{cui2021rgb,
  author = {Yingjie Cui and Qingmin Liao and Wenming Yang and Jing-Hao Xue},
  title = {{RGB Guided Depth Map Super-Resolution with Coupled U-Net}},
  booktitle = {ICME},
  year = {2021},
  doi = {10.1109/icme51207.2021.9428096},
}

Loading related content...

Figure