Viewpoint-agnostic Image Rendering

Hiroaki Aizawa, Hirokatsu Kataoka, Yutaka Satoh, Kunihito KatoWACV 2021 · OpenReview

Abstract

Rendering an any-viewpoint image is extremely difficult for Generative Adversarial Networks. This is because conventional GANs do not understand 3D information under-lying a given viewpoint image such as an object shape and relationship between viewpoint and objects in 3D space. In this paper, we present how to perform a Viewpoint-Agnostic Image Rendering (VAIR), equipping a conditional GAN with a mechanism to reconstruct 3D information of the input view. VAIR realizes any-viewpoint image generation by manipulating a viewpoint in 3D space where the reconstructed instance shape is arranged. In addition, we convert the reconstructed 3D shape into a 2D representation for image-based conditional GAN, while preserving detail 3D information. The representation consists of a depth image and 2D semantic keypoint images, which are obtained by rendering the shape from a viewpoint. In the experiment, we evaluate using a CUB-200-2011 dataset, which contains few-samples biased a viewpoint such that covers only part of the target appearance. As a result, our VAIR clearly renders an any-viewpoint image.
BibTeX
@inproceedings{aizawa2021viewpoint,
  author = {Hiroaki Aizawa and Hirokatsu Kataoka and Yutaka Satoh and Kunihito Kato},
  title = {{Viewpoint-agnostic Image Rendering}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=1rOkGdCLYoR},
}

Loading related content...

Figure