Research progress in objective quality evaluation of virtual reality images

Zhou Yu, Yi Wang, Leida Li, Chenqiang Gao, Zhaolin LuJournal of Image and Graphics 2022 · DOI

Abstract

With the rapid advancement of technology, Virtual Reality (VR) has permeated sectors ranging from healthcare and education to military and entertainment. Visual quality is the cornerstone of a successful VR experience, yet assessing it remains a significant challenge. This paper, originally published in the Journal of Image and Graphics (2022), provides a comprehensive survey of objective quality assessment (OQA) for VR images. We categorize existing models into Full-Reference (FR) and No-Reference (NR) types, further distinguishing them by their projection spaces and feature extraction methods. We also review performance metrics, existing databases, and identify future research trajectories in this burgeoning field.

Journal of Image and Graphics, Vol. 27, No. 8 (2022)

Introduction: The VR Visual Challenge

Virtual Reality aims to provide an immersive, "being there" experience. However, poor image quality can lead to "cybersickness" and break immersion. Unlike traditional 2D images, VR content is inherently spherical. To be processed by standard hardware, this spherical data is typically projected onto a 2D plane using Equirectangular Projection (ERP).

This projection introduces two major hurdles for quality assessment:

  1. Geometric Distortion: ERP maps stretch the poles significantly. A pixel at the pole represents a much smaller area on the sphere than a pixel at the equator, making standard pixel-wise metrics like PSNR inaccurate.
  2. The Viewport Constraint: Users only see a portion of the sphere (the viewport) at any given time. Traditional metrics that analyze the entire ERP map often weight areas the user never sees as heavily as those they do.

Methodologies in Objective Quality Assessment

Objective models are designed to predict human perceived quality automatically. They are broadly classified into two categories:

1. Full-Reference (FR) Models

FR models compare a distorted image against a pristine original.

  • Spherical Adaptations: Metrics like S-PSNR (Spherical PSNR) and CPP-PSNR (Craster’s Parabolic Projection PSNR) modify traditional calculations to account for the latitude-dependent pixel density of ERP maps.
  • Machine Learning Approaches: These models extract features from the ERP maps and use regression techniques (e.g., Support Vector Regression) to map these features to subjective quality scores.

2. No-Reference (NR) Models

NR models, which assess quality without a reference image, are more practical for real-world applications. They are categorized by their feature expression space:

  • ERP-based: These extract features directly from the 2D projected map, often struggling with the aforementioned geometric distortions.
  • Alternative Projections: Some models use Cube Maps (CMP) or Segmented Spherical Projections (SSP) to minimize stretching before analysis.
  • Viewport-based: These represent the current state-of-the-art. They sample multiple viewports from the sphere, analyze them as standard 2D images, and aggregate the results. This approach most closely mimics the human viewing process.

Benchmarking and Performance

The paper identifies several key databases used for training and testing, such as CVIQD, OIQA, and LIVE 3D VR IQA. Notably, newer datasets like ISIQA include "stitching distortions"—artifacts created when merging multiple camera feeds—which remain the most difficult for current algorithms to detect.

While FR models generally achieve high correlation with human scores (PLCC > 0.80 on compression tasks), NR models still lag, particularly when faced with complex, non-uniform distortions like stitching errors.

Future Directions

The field is moving toward more holistic assessment. Future research is expected to focus on:

  • Dynamic Content: Moving from static images to VR video, accounting for temporal consistency.
  • Visual Comfort: Integrating metrics for motion sickness and eye strain.
  • Saliency Integration: Using "gaze-tracking" data to weight the importance of different regions of the sphere.
  • Multi-modal Assessment: Combining visual quality with spatial audio performance for a true measure of immersion.

This summary is based on the survey "Research progress in objective quality evaluation of virtual reality images" published in the Journal of Image and Graphics, 2022.

Summary updated

Abstract

随着科学技术的发展,虚拟现实(virtual reality, VR)技术逐渐渗透到医疗、教育、军事和娱乐等众多领域,并凭借在各个领域广阔的应用前景而受到广泛关注。鉴于视觉质量是决定VR技术能否成功应用的关键,且图像是VR应用最基础和最重要的视觉信息载体,VR图像质量评价已经成为质量评价领域的重要前沿性研究方向。与传统图像质量评价类似,VR图像质量评价可以分为主观质量评价和客观质量评价。由于客观质量评价相比主观质量评价具有成本低、稳定性高和应用范围广等优点,对VR图像客观质量评价的研究受到了国内外学者的高度重视。目前,关于VR图像客观质量评价的研究已经取得了一定进展,但是文献中缺少对该方向相关研究方法的综述。基于此,本文针对VR图像客观质量评价的研究进行概述。首先,对VR图像质量评价的研究现状进行分析。然后,重点对现有的VR图像客观质量评价模型进行综述。具体地,根据模型是否需要使用原始无失真图像信息作为参考,将现有的VR图像客观质量评价模型划分为全参考型和无参考型两大类。其中,全参考型方法进一步划分为基于峰值信噪比/结构相似度的方法和基于传统机器学习的方法。根据特征表达空间的不同,无参考型VR图像质量评价模型划分为3类:基于等矩形投影表达空间的方法、基于其他投影表达空间的方法和基于实际观看空间的方法。介绍完各类模型后,分别对其相应的优缺点进行分析。此外,本文对VR图像客观质量评价模型的性能评价指标和现有VR图像质量评价数据库进行了归纳。最后对VR图像客观质量评价模型的应用进行了介绍,并指出了未来的研究可能的发展方向。
BibTeX
@article{yu2022research,
  author = {Zhou Yu and Yi Wang and Leida Li and Chenqiang Gao and Zhaolin Lu},
  title = {{Research progress in objective quality evaluation of virtual reality images}},
  journal = {Journal of Image and Graphics},
  year = {2022},
  doi = {10.11834/jig.210949},
}
Figure