Photorealistic True-Dimensional Visualization of Remote Panoramic Views for VR Headsets

Salvatore Livatino, Alessio Regalbuto, Giuseppe Morana, Giovanni Signorello, Giovanni Gallo, Alessandro Torrisi, Gianluca Padula, Katarzyna Pelc, Alessio Malizia, Giovanni Maria FarinellaIEEE Access 2023 · DOI · OpenReview

Abstract

While Virtual Reality (VR) hardware has advanced rapidly, research has predominantly focused on synthetic, computer-generated environments. This study investigates the user experience in photo-based omnidirectional environments, specifically examining how display characteristics, environmental context, and user familiarity influence the sense of "presence" and spatial accuracy. Through two user studies involving high-resolution stereoscopic panoramas of Sicilian nature reserves, the researchers demonstrate that pixel density and prior site familiarity are primary drivers of realism, often surpassing traditional metrics like color accuracy.

Introduction

The transition of VR from a niche technology to an affordable consumer product has opened new avenues for remote exploration. However, a significant gap remains in understanding how users perceive and perform within photorealistic, rather than synthetic, environments. This paper addresses the fundamental question: what makes a real-world location feel "real" when viewed through a headset?

A panoramic view of Lachea Island visualized through a VR headset setup.

The authors specifically investigate the interplay between hardware specifications (pixel density vs. lighting quality), environmental context (bright open spaces vs. dark enclosed caves), and human factors (prior familiarity with the location).

Methodology

The research utilized a smartphone-based VR setup (VR Shinecon) to maintain consistency in lens and housing variables while varying the display source. Two distinct studies were conducted:

  1. Study 1 (Display & Environment): A within-subject study ($N=20$) comparing an LG G-series display (538 ppi) against an iPhone display (326 ppi). While the iPhone offered superior color accuracy and contrast, the LG provided higher pixel density. Users viewed two Sicilian nature reserves: Lachea Island (bright, wide-open) and Monello Cave (dark, enclosed).
  2. Study 2 (Familiarity): An in-between subject study ($N=40$) comparing "Site-Familiar" users—those who visited the locations regularly—against "Unfamiliar" users.

The content consisted of high-resolution, stereoscopic-3D (S3D) panoramas captured with a Fujifilm 3D FinePix W3. Evaluation metrics included the Igroup Presence Questionnaire (IPQ), subjective emotional ratings, and objective egocentric distance estimation tasks ranging from 2 to 9.2 meters.

Key Insights: Resolution and the "Transparency" Effect

A primary finding of the study is that resolution often trumps color fidelity in photorealistic VR. Despite the iPhone's objectively better technical specifications for black levels and contrast, the LG display’s higher pixel density led to significantly higher perceived realism and better "image lighting" scores ($p = 0.0482$). This suggests that the human brain prioritizes fine detail and texture over color accuracy when processing photographic environments.

Furthermore, the study identifies a "transparency" effect: once a display reaches a certain quality threshold, the hardware becomes secondary to the content. At this point, the user's emotional state and sense of presence are dictated almost entirely by the environment itself. For instance, Lachea Island consistently triggered feelings of happiness and relaxation, whereas Monello Cave induced anxiety ($p = 0.0031$) and a heightened sense of "acting there" due to the perceived proximity of the cave walls.

The Role of Familiarity and Spatial Competence

User familiarity acted as a significant "presence multiplier." Participants who had visited the locations in real life reported higher realism scores ($p = 0.0458$). Their brains appeared to "fill in the gaps" of the static images with sensory memories, suggesting that VR is most effective for remote visits when the user has some baseline context of the space.

Remarkably, the photorealistic setup allowed for high spatial competence. Unlike synthetic VR, where depth perception is often skewed, users in this study estimated distances with errors frequently below 5-8%. This confirms that high-fidelity S3D photography can successfully mimic natural binocular vision, making it a viable tool for high-precision tele-exploration.

Conclusion and Future Directions

This research highlights that for photorealistic VR, pixel density and user familiarity are more critical than traditional display metrics like color gamut. While the study was limited by the use of static 3-Degrees-of-Freedom (3DOF) panoramas, it provides a foundation for future work involving dynamic 3D video and 6DOF motion. Future research will explore the "uncanny valley" threshold between high-end photorealistic textures and state-of-the-art synthetic environments to further refine the parameters of spatial presence.

Summary updated

Abstract

Virtual Reality headsets have evolved to include unprecedented display quality. Meantime, they have become light-weight, wireless and low-cost, which has opened to new applications and a much wider audience. Photo-based omnidirectional imaging has also developed, becoming directly exploitable for VR, with their combination proven suitable for: remote visits and realistic scene reconstruction, operator’s training and control panels, surveillance and e-tourism. There is however a limited amount of scientific work assessing VR experience and user’s performance in photo-based environment representations. This paper focuses on assessing the effect of photographic realism in VR when observing real places through a VR headset, for two different pixel-densities of the display, environment types and familiarity levels. Our comparison relies on the observation of static three-dimensional and omnidirectional photorealistic views of environments. The aim is to gain an insight about how photographic texture can affect perceived realness, sense of presence and provoked emotions, as well as perception of image-lighting and actual space dimension (true-dimension). Two user studies are conducted based on subjective rating and measurements given by users to a number of display and human factors. The display pixel-density affected the perceived image-lighting and prevailed over better lighting specs. The environment illumination and distance to objects generally played a stronger role than display. The environment affected the perceived image-lighting, spatial presence, depth impression and specific emotions. Distances to a set of objects were generally accurately estimated. Place familiarity enhanced perceived realism and presence. They confirmed some previous studies, but also introduced new elements.
BibTeX
@article{livatino2023photorealistic,
  author = {Salvatore Livatino and Alessio Regalbuto and Giuseppe Morana and Giovanni Signorello and Giovanni Gallo and Alessandro Torrisi and Gianluca Padula and Katarzyna Pelc and Alessio Malizia and Giovanni Maria Farinella},
  title = {{Photorealistic True-Dimensional Visualization of Remote Panoramic Views for VR Headsets}},
  journal = {IEEE Access},
  year = {2023},
  doi = {10.1109/access.2023.3285709},
}
Figure