Weakly supervised 2D human pose transfer

Qian Zheng, Yajie Liu, Zhizhao Lin, Dani Lischinski, Daniel Cohen-Or, Hui Huang

Abstract

We present a novel method for pose transfer between two 2D human skeletons. When the bone lengths and proportions between the two skeletons are significantly different, pose transfer becomes a challenging task, which cannot be accomplished by simply copying the joint positions or the bone directions. Our data-driven approach utilizes a deep neural network trained, in a weakly supervised fashion, to encode a skeleton into two separate latent codes, one representing its pose, and another representing the skeleton’s proportions (skeleton-ID). The network is given two skeletons, and learns to combine the pose of one with the skeleton-ID of the other. Lacking supervision on the poses, we develop a novel loss that qualitatively compares poses of different skeletons. We evaluate the performance of our method on a large set of poses. The advantages of avoiding supervision are demonstrated by showing transfer of extreme poses, as well as between uncommon skeleton proportions.
BibTeX
@article{zheng2021weakly,
  author = {Qian Zheng and Yajie Liu and Zhizhao Lin and Dani Lischinski and Daniel Cohen-Or and Hui Huang},
  title = {{Weakly supervised 2D human pose transfer}},
  journal = {Science China Information Sciences},
  year = {2021},
  doi = {10.1007/s11432-021-3301-5},
}

Loading related content...

Figure