Weakly-Supervised Single-view Dense 3D Point Cloud Reconstruction via Differentiable Renderer

Peng Jin, Shaoli Liu, Jianhua Liu, Hao Huang, Linlin Yang, Michael Weinmann, Reinhard KleinChinese 2021 · DOI

Abstract

Abstract In recent years, addressing ill-posed problems by leveraging prior knowledge contained in databases on learning techniques has gained much attention. In this paper, we focus on complete three-dimensional (3D) point cloud reconstruction based on a single red-green-blue (RGB) image, a task that cannot be approached using classical reconstruction techniques. For this purpose, we used an encoder-decoder framework to encode the RGB information in latent space, and to predict the 3D structure of the considered object from different viewpoints. The individual predictions are combined to yield a common representation that is used in a module combining camera pose estimation and rendering, thereby achieving differentiability with respect to imaging process and the camera pose, and optimization of the two-dimensional prediction error of novel viewpoints. Thus, our method allows end-to-end training and does not require supervision based on additional ground-truth (GT) mask annotations or ground-truth camera pose annotations. Our evaluation of synthetic and real-world data demonstrates the robustness of our approach to appearance changes and self-occlusions, through outperformance of current state-of-the-art methods in terms of accuracy, density, and model completeness.
BibTeX
@inproceedings{jin2021weakly,
  author = {Peng Jin and Shaoli Liu and Jianhua Liu and Hao Huang and Linlin Yang and Michael Weinmann and Reinhard Klein},
  title = {{Weakly-Supervised Single-view Dense 3D Point Cloud Reconstruction via Differentiable Renderer}},
  booktitle = {Chinese},
  year = {2021},
  doi = {10.1186/s10033-021-00615-x},
}

Loading related content...

Figure