Abstract
With the rapid advancement of technology, Virtual Reality (VR) has permeated sectors ranging from healthcare and education to military and entertainment. Visual quality is the cornerstone of a successful VR experience, yet assessing it remains a significant challenge. This paper, originally published in the Journal of Image and Graphics (2022), provides a comprehensive survey of objective quality assessment (OQA) for VR images. We categorize existing models into Full-Reference (FR) and No-Reference (NR) types, further distinguishing them by their projection spaces and feature extraction methods. We also review performance metrics, existing databases, and identify future research trajectories in this burgeoning field.
Introduction: The VR Visual Challenge
Virtual Reality aims to provide an immersive, "being there" experience. However, poor image quality can lead to "cybersickness" and break immersion. Unlike traditional 2D images, VR content is inherently spherical. To be processed by standard hardware, this spherical data is typically projected onto a 2D plane using Equirectangular Projection (ERP).
This projection introduces two major hurdles for quality assessment:
- Geometric Distortion: ERP maps stretch the poles significantly. A pixel at the pole represents a much smaller area on the sphere than a pixel at the equator, making standard pixel-wise metrics like PSNR inaccurate.
- The Viewport Constraint: Users only see a portion of the sphere (the viewport) at any given time. Traditional metrics that analyze the entire ERP map often weight areas the user never sees as heavily as those they do.
Methodologies in Objective Quality Assessment
Objective models are designed to predict human perceived quality automatically. They are broadly classified into two categories:
1. Full-Reference (FR) Models
FR models compare a distorted image against a pristine original.
- Spherical Adaptations: Metrics like S-PSNR (Spherical PSNR) and CPP-PSNR (Craster’s Parabolic Projection PSNR) modify traditional calculations to account for the latitude-dependent pixel density of ERP maps.
- Machine Learning Approaches: These models extract features from the ERP maps and use regression techniques (e.g., Support Vector Regression) to map these features to subjective quality scores.
2. No-Reference (NR) Models
NR models, which assess quality without a reference image, are more practical for real-world applications. They are categorized by their feature expression space:
- ERP-based: These extract features directly from the 2D projected map, often struggling with the aforementioned geometric distortions.
- Alternative Projections: Some models use Cube Maps (CMP) or Segmented Spherical Projections (SSP) to minimize stretching before analysis.
- Viewport-based: These represent the current state-of-the-art. They sample multiple viewports from the sphere, analyze them as standard 2D images, and aggregate the results. This approach most closely mimics the human viewing process.
Benchmarking and Performance
The paper identifies several key databases used for training and testing, such as CVIQD, OIQA, and LIVE 3D VR IQA. Notably, newer datasets like ISIQA include "stitching distortions"—artifacts created when merging multiple camera feeds—which remain the most difficult for current algorithms to detect.
While FR models generally achieve high correlation with human scores (PLCC > 0.80 on compression tasks), NR models still lag, particularly when faced with complex, non-uniform distortions like stitching errors.
Future Directions
The field is moving toward more holistic assessment. Future research is expected to focus on:
- Dynamic Content: Moving from static images to VR video, accounting for temporal consistency.
- Visual Comfort: Integrating metrics for motion sickness and eye strain.
- Saliency Integration: Using "gaze-tracking" data to weight the importance of different regions of the sphere.
- Multi-modal Assessment: Combining visual quality with spatial audio performance for a true measure of immersion.
This summary is based on the survey "Research progress in objective quality evaluation of virtual reality images" published in the Journal of Image and Graphics, 2022.
Summary updated