Color-Guided Depth Recovery via Joint Local Structural and Nonlocal Low-Rank Regularization

Weisheng Dong, Guangming Shi, Xin Li, Kefan Peng, Jinjian Wu, Zhenhua Guo

Abstract

High-quality depth recovery from RGB-D data has received increasingly more attention in recent years due to their wide applications from depth-based image rendering to three-dimensional imaging and video. Sharp contrast between high-quality color images and low-quality depth maps presents severe challenges to the development of color-guided depth recovery techniques. Previous works have emphasized either locally varying characteristics of color-depth dependence or nonlocal similarities around the discontinuities of the scene geometry. Therefore, it is desirable to exploit both local and nonlocal structural constraints for optimizing the performance of color-guided depth recovery. In this work, we propose a unified variational approach via joint local and nonlocal regularization. The local regularization term consists of two complementary parts-one characterizing the color-depth dependence in the gradient domain and the other in the spatial domain; nonlocal regularization involves a low-rank constraint suitable for large-scale depth discontinuities. Extensive experimental results are reported to show that our approach outperforms several existing state-of-the-art depth recovery methods on both synthetic and real-world data sets.
BibTeX
@article{dong2017color,
  author = {Weisheng Dong and Guangming Shi and Xin Li and Kefan Peng and Jinjian Wu and Zhenhua Guo},
  title = {{Color-Guided Depth Recovery via Joint Local Structural and Nonlocal Low-Rank Regularization}},
  journal = {IEEE Trans. Multimedia},
  year = {2017},
  doi = {10.1109/tmm.2016.2613824},
}

Loading related content...

Figure