Spherical Image Generation from a Single Normal Field of View Image by Considering Scene Symmetry

Takayuki Hara, Tatsuya HaradaarXiv 2020 · OpenReview

Abstract: Spherical images taken in all directions (360 degrees) allow representing the surroundings of the subject and the space itself, providing an immersive experience to the viewers. Generating a spherical image from a single normal-field-of-view (NFOV) image is convenient and considerably expands the usage scenarios because there is no need to use a specific panoramic camera or take images from multiple directions; however, it is still a challenging and unsolved problem. The primary challenge is controlling the high degree of freedom involved in generating a wide area that includes the all directions of the desired plausible spherical image. On the other hand, scene symmetry is a basic property of the global structure of the spherical images, such as rotation symmetry, plane symmetry and asymmetry. We propose a method to generate spherical image from a single NFOV image, and control the degree of freedom of the generated regions using scene symmetry. We incorporate scene-symmetry parameters as latent variables into conditional variational autoencoders, following which we learn the conditional probability of spherical images for NFOV images and scene symmetry. Furthermore, the probability density functions are represented using neural networks, and scene symmetry is implemented using both circular shift and flip of the hidden variables. Our experiments show that the proposed method can generate various plausible spherical images, controlled from symmetric to asymmetric.

1. Introduction

Generating a full 360-degree spherical image from a single "normal" field-of-view (NFOV) image—such as a standard smartphone photo—is an ill-posed problem. Because the input captures only a small fraction of the environment, infinite possibilities exist for completing the scene. Existing image-completion methods often struggle with structural discontinuities (seams) at the edges of spherical projections and a lack of global structural control, leading to chaotic or impossible environments.

This paper introduces a novel method to transform a single NFOV photo into a complete 360-degree spherical image by incorporating "scene symmetry" as a controllable latent variable. By leveraging symmetry as a structural prior, the authors provide a mechanism to guide the generation process, producing diverse and structurally plausible environments that range from perfectly symmetric to naturally asymmetric.

2. Methodology

The proposed framework utilizes a Conditional Variational Autoencoder (CVAE) that treats scene symmetry as a first-class citizen.

Overview of different scene symmetry types used as structural priors: rotational symmetry (90°, 180°, all-round), plane symmetry (0°, 90°), and asymmetry.
  • Symmetry as a Latent Variable: The model uses two latent variables: $z$ for general content and $s$ for symmetry. The symmetry parameter $s$ allows users to explicitly control the type and intensity of symmetry in the output (e.g., 90° rotational symmetry vs. plane symmetry).
  • Symmetry Operations: To implement these constraints, the authors use circular shifts (representing rotation around the vertical axis) and flips (representing plane symmetry) of the hidden feature maps within the network.
  • Circular Padding: To solve the "seam" problem, standard zero-padding in CNN layers is replaced with circular padding. This ensures the network recognizes that the left and right edges of the equirectangular map connect, maintaining continuity across the sphere.
  • Symmetry-Control Function ($H$): This function calculates a weighted linear sum of symmetric transformations, applying higher weights to features near the original input and attenuating them as they move around the sphere to ensure a natural blend.

3. Experimental Setup

The model was trained and evaluated using the SUN360 dataset, a large-scale collection of spherical panoramas. The dataset included 50,000 images for training, 10,000 for testing, and 5,000 for validation. All images were converted to a $256 \times 512$ equirectangular format. NFOV input images were simulated by cropping the panoramas to a field-of-view between 30° and 120°.

4. Results and Discussion

The proposed method significantly outperformed state-of-the-art baselines like Pix2pix and PICNet. Quantitatively, the model achieved a Fréchet Inception Distance (FID) score of 28.6, a substantial improvement over PICNet (39.6) and Pix2pix (46.3).

Example of a generated spherical image exhibiting 90° rotational symmetry in an architectural setting.

Qualitatively, the model demonstrates an ability to take a single photo and generate structurally consistent environments, such as the 90-degree rotational room shown above, while maintaining sharp textures and consistent lighting. A key insight is that the symmetry parameter $s$ is continuous, allowing the model to generate "mostly symmetric" rooms that include realistic, slight asymmetries (e.g., a misplaced chair), making the scenes look natural rather than artificial. Ablation studies confirmed that both circular padding and the symmetry control function are vital for global consistency.

5. Conclusion

While the model excels at global structural control, it currently operates at a relatively low resolution ($256 \times 512$), which limits its use in high-end VR applications. Additionally, performance drops when the input image has a very narrow field of view (e.g., 30°), as there is insufficient visual information to reliably predict the global structure. Future work will focus on generating high-resolution images and providing finer control over local object placement.

Summary updated

Abstract

Spherical images taken in all directions (360 degrees) allow representing the surroundings of the subject and the space itself, providing an immersive experience to the viewers. Generating a spherical image from a single normal-field-of-view (NFOV) image is convenient and considerably expands the usage scenarios because there is no need to use a specific panoramic camera or take images from multiple directions; however, it is still a challenging and unsolved problem. The primary challenge is controlling the high degree of freedom involved in generating a wide area that includes the all directions of the desired plausible spherical image. On the other hand, scene symmetry is a basic property of the global structure of the spherical images, such as rotation symmetry, plane symmetry and asymmetry. We propose a method to generate spherical image from a single NFOV image, and control the degree of freedom of the generated regions using scene symmetry. We incorporate scene-symmetry parameters as latent variables into conditional variational autoencoders, following which we learn the conditional probability of spherical images for NFOV images and scene symmetry. Furthermore, the probability density functions are represented using neural networks, and scene symmetry is implemented using both circular shift and flip of the hidden variables. Our experiments show that the proposed method can generate various plausible spherical images, controlled from symmetric to asymmetric.
BibTeX
@misc{hara2020spherical,
  author = {Takayuki Hara and Tatsuya Harada},
  title = {{Spherical Image Generation from a Single Normal Field of View Image by Considering Scene Symmetry}},
  year = {2020},
  eprint = {2001.02993},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2001.02993},
}
Figure