Blind Omnidirectional Image Quality Assessment: Integrating Local Statistics and Global Semantics

Wei Zhou, Zhou Wang

Abstract

Omnidirectional image quality assessment (OIQA) aims to predict the perceptual quality of omnidirectional images that cover the whole 180$×$360$^{∘}$ viewing range of the visual environment. Here we propose a blind/no-reference OIQA method named $S^2$ that bridges the gap between low-level statistics and high-level semantics of omnidirectional images. Specifically, statistic and semantic features are extracted in separate paths from multiple local viewports and the hallucinated global omnidirectional image, respectively. A quality regression along with a weighting process is then followed that maps the extracted quality-aware features to a perceptual quality prediction. Experimental results demonstrate that the proposed $S^2$ method offers highly competitive performance against state-of-the-art methods.

Introduction

Omnidirectional Image Quality Assessment (OIQA) presents unique challenges compared to standard 2D image assessment. In a Virtual Reality (VR) environment, a viewer never perceives the entire 360-degree field of view simultaneously; instead, they browse through various "viewports." Existing metrics often suffer from two primary limitations: a dependency on reference images (Full-Reference) and a "semantic gap." Most models focus exclusively on low-level pixel statistics (e.g., blur, noise) or high-level content recognition, whereas human perception naturally integrates both. For instance, a human observer identifies a scene as a "classroom" while simultaneously noticing if that scene appears grainy or distorted.

Comparison between previous models and the proposed S² method, highlighting the integration of semantic reconstruction.

The $S^2$ Framework

To address these challenges, the researchers developed the $S^2$ framework, which mimics human VR browsing by utilizing two parallel processing paths:

1. The Statistic Path (Local)

This path focuses on low-level visual details. The model samples $N$ local viewports from the 360-degree image and constructs multi-scale Gaussian and Laplacian pyramids. From these, it extracts:

  • Local Binary Patterns (LBP): To capture texture and structural statistics.
  • Natural Scene Statistics (NSS): To identify deviations from the expected "natural" appearance caused by distortions.

2. The Semantic Path (Global)

This path evaluates the high-level scene context. By analyzing the "hallucinated" global omnidirectional image (the projected map), the model uses a pre-trained VGG-M network to extract high-level semantic features.

The architecture of the proposed S² framework, featuring parallel Statistic and Semantic paths.

Feature Fusion

Features from both paths are processed through separate Support Vector Regression (SVR) models. The resulting scores are combined via a weighted average to produce the final perceptual quality prediction.

Experimental Evaluation

The model was evaluated using the CVIQD (Compressed VR Image Quality Database), which contains 16 original images and 528 distorted versions compressed via JPEG, AVC (H.264), and HEVC (H.265). The benchmark for success was the Mean Opinion Scores (MOS) provided by human subjects.

Key Results

The $S^2$ method outperformed all tested No-Reference (NR) models and several Full-Reference (FR) models on the CVIQD dataset:

  • SROCC (Spearman Correlation): 0.9710
  • PLCC (Pearson Correlation): 0.9781
  • RMSE (Root Mean Squared Error): 2.8945

Notably, it surpassed the previous state-of-the-art NR-OIQA model, MFILGN, which achieved an SROCC of 0.9670.

Discussion and Insights

  • Semantic Confidence: A key finding is that as distortion increases, the model's "confidence" in recognizing the scene decreases. This mirrors human frustration when heavy noise or blur obscures meaningful content.
  • Encoder Performance: The study found that semantic confidence drops more slowly in HEVC-compressed images compared to JPEG or AVC, suggesting HEVC better preserves scene structures at lower bitrates.
  • Efficiency: The framework is computationally efficient; using just 6 sampled viewports yielded performance nearly identical to using 80.
  • Complementary Paths: Ablation studies confirmed that while the Statistic Path is the primary driver of quality prediction, the Semantic Path provides critical complementary information that achieves state-of-the-art accuracy.

Conclusion

The $S^2$ framework successfully bridges the gap between low-level statistics and high-level semantics for blind OIQA. While the current model is highly effective, future work may involve exploring different deep learning backbones for the semantic path and extending the dual-path approach to 360-degree video to account for temporal statistics.

Summary updated

Abstract

Omnidirectional image quality assessment (OIQA) aims to predict the perceptual quality of omnidirectional images that cover the whole 180$\times$360$^{\circ}$ viewing range of the visual environment. Here we propose a blind/no-reference OIQA method named S$^2$ that bridges the gap between low-level statistics and high-level semantics of omnidirectional images. Specifically, statistic and semantic features are extracted in separate paths from multiple local viewports and the hallucinated global omnidirectional image, respectively. A quality regression along with a weighting process is then followed that maps the extracted quality-aware features to a perceptual quality prediction. Experimental results demonstrate that the proposed S$^2$ method offers highly competitive performance against state-of-the-art methods.
BibTeX
@inproceedings{zhou2023blind,
  author = {Wei Zhou and Zhou Wang},
  title = {{Blind Omnidirectional Image Quality Assessment: Integrating Local Statistics and Global Semantics}},
  booktitle = {ICIP},
  year = {2023},
  doi = {10.1109/icip49359.2023.10222049},
}
Figure