Learning Monocular Mesh Recovery of Multiple Body Parts Via Synthesis

Yu Sun, Tianyu Huang, Qian Bao, Wu Liu, Wenpeng Gao, Yili FuICASSP 2022 · OpenReview

Abstract

In this paper, we focus on simultaneously recovering the 3D mesh of multiple body parts from a single RGB image. One of the main challenges is that available datasets with full-body 3D annotations are very limited. This results in poor generalization ability of existing learning-based methods. Existing optimization-based methods iteratively fit the 3D mesh to the 2d pose, which is very time-consuming. To address these limitations, we propose to integrate multiple 3D single-body-part datasets to create a highly diverse whole-body 3D motion space for learning from controllable synthetics. Compared with the learning-based approaches, the proposed method greatly alleviates the reliance on training data. Compared with the optimization-based approaches, the proposed method is a hundred times faster. Our proposed method also outperforms previous state-of-the-art methods on CMU Panoptic dataset.
BibTeX
@inproceedings{sun2022learning,
  author = {Yu Sun and Tianyu Huang and Qian Bao and Wu Liu and Wenpeng Gao and Yili Fu},
  title = {{Learning Monocular Mesh Recovery of Multiple Body Parts Via Synthesis}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=h1PQtW9Mxgw},
}

Loading related content...

Figure