Abstract
Predicting the quality of 360-degree omnidirectional images (ODIs) is a critical task for Virtual Reality (VR) applications, yet it is complicated by high resolutions, projection-induced distortions, and the frequent absence of high-quality reference images. This paper proposes SAP-net, a novel no-reference (NR) quality assessment tool that leverages self-attention to identify perceptually significant regions without the need for costly human eye-tracking data. By utilizing a Wavelet-based Residual Enhancement module to create a pseudo-reference and Residual Spatial Attention Blocks for quality estimation, SAP-net achieves state-of-the-art performance. Experimental results on the newly introduced IQA-ODI dataset demonstrate that SAP-net significantly outperforms existing NR methods and even surpasses several full-reference metrics in consistency with human subjective scores.
Introduction
Omnidirectional images (ODIs) are foundational to immersive VR experiences but present unique challenges for Image Quality Assessment (IQA). These images are typically massive (8K or higher) and must be projected from a spherical format onto a 2D plane, introducing geometric distortions. Furthermore, heavy compression is often required for streaming, which further degrades quality. Traditional metrics like PSNR or SSIM require a "perfect" reference image, which is rarely available in live streaming or user-generated content scenarios. Existing NR methods often rely on explicit human saliency maps—data indicating where viewers look—which is expensive and time-consuming to collect. SAP-net addresses these issues by implicitly learning human attention through a self-attention mechanism, reducing computational overhead and eliminating the need for saliency labels.
Methods
The SAP-net architecture consists of three primary components designed to mimic the human visual system's response to distortions:
- Wavelet-based Residual Enhancement (WBRE): Standard pooling layers often discard high-frequency details essential for quality assessment. WBRE uses the Discrete Wavelet Transform (DWT) to attempt a reconstruction of a high-quality version of the impaired image, serving as a "pseudo-reference."
- Perceptual Quality Estimation (PQE): This module compares the impaired image with the pseudo-reference to generate an error map. It employs Residual Spatial Attention Blocks (RSAB) to perform self-attention, allowing the network to focus on regions most likely to impact human perception without requiring external eye-tracking data.
- Quality Regression (QR): The final stage integrates the perceptual quality map and error map to output a quality score consistent with human perception.
The IQA-ODI Dataset
To train and validate the model, the authors introduced the IQA-ODI benchmark. This dataset contains 1,080 images, including 120 high-quality 8K references and 960 impaired versions. Impairments include four levels of JPEG compression applied across four different projection patterns: Cube Map (CMP), Icosahedral (ISP), Octahedral (OHP), and Craster Parabolic (CPP). Subjective ground truth was established by 200 participants using HTC Vive VR headsets, providing Differential Mean Opinion Scores (DMOS).
Results and Discussion
SAP-net was benchmarked against nine state-of-the-art methods. It achieved a Pearson Linear Correlation Coefficient (PLCC) of 0.9258 and a Spearman Rank-Order Correlation Coefficient (SROCC) of 0.9036, outperforming the next best NR method, dipIQ (PLCC 0.8741).
A key insight from the IQA-ODI dataset analysis is that JPEG compression levels have a far more substantial impact on perceived quality than the specific geometric projection used. As shown in Figure 4, the distribution of DMOS remains relatively consistent across different projection types, suggesting that the artifacts introduced by compression are the primary drivers of quality degradation in VR environments.
Conclusion
SAP-net provides a highly efficient and accurate solution for no-reference 360-degree image quality assessment. By replacing standard pooling with DWT and utilizing implicit spatial attention, it avoids the need for expensive saliency data and complex viewport extraction. Future research will focus on extending this spatial attention mechanism to the temporal domain for 360-degree video and expanding the dataset to include a wider variety of real-world distortions such as sensor noise and transmission loss.
Summary updated