Tensor Oriented No-Reference Light Field Image Quality Assessment

Wei Zhou, Likun Shi, Zhibo Chen

Abstract

Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media. However, due to the inherent high dimensional characteristics of LFI, the LFI quality assessment turns into a multi-dimensional problem that requires consideration of the quality degradation in both spatial and angular dimensions. Therefore, we propose a novel Tensor oriented No-reference Light Field image Quality evaluator (Tensor-NLFQ) based on tensor theory. Specifically, since the LFI is regarded as a low-rank 4D tensor, the principal components of four oriented sub-aperture view stacks are obtained via Tucker decomposition. Then, the Principal Component Spatial Characteristic (PCSC) is designed to measure the spatial-dimensional quality of LFI considering its global naturalness and local frequency properties. Finally, the Tensor Angular Variation Index (TAVI) is proposed to measure angular consistency quality by analyzing the structural similarity distribution between the first principal component and each view in the view stack. Extensive experimental results on four publicly available LFI quality databases demonstrate that the proposed Tensor-NLFQ model outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.

Introduction

Light field images (LFIs) capture both the intensity and direction of light, enabling post-capture refocusing and perspective shifts. This 4D data structure ($x, y, s, t$) introduces unique artifacts that traditional 2D metrics cannot capture, particularly "angular consistency"—the smoothness of transitions between viewpoints. Existing 3D or multi-view metrics often overlook the intrinsic high-dimensional correlations of the light field. Furthermore, most no-reference (NR) metrics ignore chrominance distortion, which significantly impacts perceived quality in immersive media.

Methodology: The Tensor-NLFQ Pipeline

The proposed Tensor-NLFQ model treats the light field as a unified high-dimensional object rather than a collection of independent 2D images. The pipeline consists of five main stages:

  1. Color Space Conversion: RGB input is converted to CIELAB ($L^*, a^*, b^*$) to better align with human perception of luminance and chrominance.
  2. View Stacking: Sub-aperture images (SAIs) are grouped into stacks along four orientations (0°, 45°, 90°, 135°) to capture angular inconsistencies in all directions.
  3. Tucker Decomposition: Each 3D view stack is decomposed to reduce angular redundancy. The first principal component, containing over 70% of the energy, is extracted as the primary spatial structure.
  4. Feature Extraction:
    • Principal Component Spatial Characteristic (PCSC): Measures spatial quality using natural scene statistics (NSS) and Discrete Cosine Transform (DCT) entropy.
  5. Tensor Angular Variation Index (TAVI): Measures angular consistency by calculating the structural similarity (SSIM) between the first principal component and every other view in the stack.
  6. Quality Prediction: These features are mapped to a final quality score using a Support Vector Regression (SVR) model.
The overall flow diagram of the Tensor-NLFQ model, illustrating the transition from raw LFI to predicted quality score.

Experimental Evaluation

The model was benchmarked against four major public databases: Win5-LID, MPI-LFA, SMART, and VALID. Tensor-NLFQ consistently outperformed state-of-the-art 2D, 3D, and LFI-specific metrics across various distortion types, including HEVC compression and linear interpolation artifacts.

  • Accuracy: On the Win5-LID database, it achieved a Spearman Rank Correlation Coefficient (SRCC) of 0.9101, significantly higher than the previous best of 0.8719 (BELIF).
  • Consistency: The model showed high performance across both synthetic and real-world scenes, demonstrating its robustness.

Key Insights

A central innovation is treating LFIs as 4D tensors, which is mathematically more robust than independent 2D analysis. The first principal component from Tucker decomposition serves as a "pseudo-reference," allowing the model to detect inconsistencies without the original image. Additionally, the inclusion of chrominance channels ($a^*, b^*$) proved vital, as they are highly sensitive to depth-based rendering artifacts that luminance-only models might miss.

Visual representation of the light field image components across different channels and decomposition stages.

Conclusion

Tensor-NLFQ provides a robust, efficient, and accurate no-reference metric for LFIs by leveraging tensor theory to address high-dimensional quality degradation. Future work will explore extending this tensor-based framework to light field video, where temporal consistency adds another dimension of complexity, and integrating the metric into optimization loops for LFI compression and reconstruction algorithms.

Summary updated

Abstract

Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media. However, due to the inherent high dimensional characteristics of LFI, the LFI quality assessment turns into a multi-dimensional problem that requires consideration of the quality degradation in both spatial and angular dimensions. Therefore, we propose a novel Tensor oriented No-reference Light Field image Quality evaluator (Tensor-NLFQ) based on tensor theory. Specifically, since the LFI is regarded as a low-rank 4D tensor, the principal components of four oriented sub-aperture view stacks are obtained via Tucker decomposition. Then, the Principal Component Spatial Characteristic (PCSC) is designed to measure the spatial-dimensional quality of LFI considering its global naturalness and local frequency properties. Finally, the Tensor Angular Variation Index (TAVI) is proposed to measure angular consistency quality by analyzing the structural similarity distribution between the first principal component and each view in the view stack. Extensive experimental results on four publicly available LFI quality databases demonstrate that the proposed Tensor-NLFQ model outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.
BibTeX
@article{zhou2020tensor,
  author = {Wei Zhou and Likun Shi and Zhibo Chen},
  title = {{Tensor Oriented No-Reference Light Field Image Quality Assessment}},
  journal = {IEEE Trans. Image Process.},
  year = {2020},
  url = {https://openreview.net/forum?id=GunVvYkmZ6e},
}
Figure