Center-Emphasized Visual Saliency and a Contrast-Based Full Reference Image Quality Index

Md. Abu Layek, A. F. M. Shahab Uddin, Tuyen Pham Le, TaeChoong Chung, Eui-Nam Huh

TL;DR

This paper introduces CEQI, a novel full-reference image quality assessment (FR-IQA) method that leverages visual saliency and contrast, with a specific emphasis on the image center. By giving more weight to distortions in the central region, CEQI aims to better align with human perception and achieves state-of-the-art performance on benchmark datasets while maintaining efficient processing speeds.

What Problem Does This Paper Address?

Objective Image Quality Assessment (IQA) is crucial for automating quality control in multimedia applications. While many IQA methods exist, they often struggle to accurately reflect human perception of image quality. This paper highlights a key aspect of human visual perception that has been largely overlooked: a "center bias." Psychological studies show that human eyes are naturally drawn to the center of an image and the screen. Consequently, distortions in the central region are perceived more intensely and impact overall quality assessment more significantly than identical distortions in peripheral areas. Existing IQA methods, by treating all image regions equally, fail to account for this perceptual phenomenon, leading to a discrepancy between objective scores and human judgments.

Methods

The proposed method, CEQI (Center-Emphasized Quality Index), builds upon existing IQA techniques by incorporating a novel center-emphasis strategy. The core components are:

  • Visual Saliency Similarity (VSS): The paper utilizes the spectral residual saliency approach to generate saliency maps for both reference and distorted images. A similarity metric is then computed between these saliency maps.
  • Contrast Similarity (CS): The RMS contrast is calculated for both reference and distorted images, and a similarity metric is derived from these contrast maps.
  • Center Emphasis: This is the key innovation. The authors extract the central region of the image (specifically, the 5th block in a 3x3 grid).
  • For the visual saliency maps, the similarity in the center region is amplified by element-wise multiplication with the saliency map of the central region.
  • For contrast similarity, the central region's contrast is enhanced through element-wise squaring.
  • Standard Deviation Pooling: The final quality score is obtained by calculating the standard deviation of the updated (center-emphasized) saliency and contrast similarity maps, and then combining these standard deviations using learned weights (w1 and w2). This pooling strategy is chosen for its proven effectiveness in previous IQA research.

Datasets

The proposed CEQI method was evaluated on three widely-used benchmark databases for IQA research:

  • TID2008: Contains 1700 distorted images across 17 distortion types, with 838 subjects.
  • CSIQ: Features 866 distorted images with 6 distortion types, assessed by 35 subjects.
  • LIVE: Comprises 779 distorted images across 5 distortion types, evaluated by 161 subjects.

The paper does not introduce a new dataset but rather validates its method on these established benchmarks.

Key Insights and Ideas

  • Center Bias is Crucial for Perceptual IQA: The most significant insight is that human visual perception is not uniform across an image. The inherent tendency to focus on the center significantly influences how distortions are perceived, and IQA methods must account for this to achieve better correlation with human judgments.
  • Combining Saliency, Contrast, and Center Emphasis is Powerful: The CEQI method demonstrates that a synergistic approach, integrating visual saliency and contrast with a specific focus on the central region, yields superior performance compared to methods that treat all image regions equally or rely on single features.
  • Efficiency Matters: While accuracy is paramount, the paper also emphasizes the importance of computational efficiency. CEQI achieves a good balance, offering state-of-the-art performance with a processing speed suitable for real-time applications.

Key Results

  • State-of-the-Art Performance: CEQI consistently outperforms 13 other state-of-the-art IQA methods across the TID2008, CSIQ, and LIVE databases, achieving higher correlations (PLCC, SROCC, KROCC) and lower RMSE values. For instance, on the TID2008 dataset, CEQI ranks first in overall performance across multiple metrics.
  • Consistent Performance Across Distortions: The paper highlights that CEQI provides consistent scores across various distortion types, indicating it is not overly biased towards specific artifacts.
  • Efficient Processing: CEQI can process approximately 65.51 images per second, making it suitable for real-time applications, a significant advantage over some other high-performing but slower methods like VIF.

Limitations and Future Work

  • Limitations: While the paper demonstrates strong performance, it primarily focuses on full-reference IQA. The authors acknowledge that their method's effectiveness might vary for different types of distortions or image content not well-represented in the tested databases. The specific weighting factors (w1, w2) are empirically determined and might require fine-tuning for other contexts.
  • Future Work: The authors suggest that the concept of center emphasis could be integrated into other existing IQA models, including no-reference and reduced-reference approaches, to potentially enhance their performance. They also express interest in investigating these possibilities further.

Summary updated

Abstract

Objective image quality assessment (IQA) is imperative in the current multimedia-intensive world, in order to assess the visual quality of an image at close to a human level of ability. Many~parameters such as color intensity, structure, sharpness, contrast, presence of an object, etc., draw human attention to an image. Psychological vision research suggests that human vision is biased to the center area of an image and display screen. As a result, if the center part contains any visually salient information, it draws human attention even more and any distortion in that part will be better perceived than other parts. To the best of our knowledge, previous IQA methods have not considered this fact. In this paper, we propose a full reference image quality assessment (FR-IQA) approach using visual saliency and contrast; however, we give extra attention to the center by increasing the sensitivity of the similarity maps in that region. We evaluated our method on three large-scale popular benchmark databases used by most of the current IQA researchers (TID2008, CSIQ~and LIVE), having a total of 3345 distorted images with 28~different kinds of distortions. Our~method is compared with 13 state-of-the-art approaches. This comparison reveals the stronger correlation of our method with human-evaluated values. The prediction-of-quality score is consistent for distortion specific as well as distortion independent cases. Moreover, faster processing makes it applicable to any real-time application. The MATLAB code is publicly available to test the algorithm and can be found online at http://layek.khu.ac.kr/CEQI.
BibTeX
@article{layek2019center,
  author = {Md. Abu Layek and A. F. M. Shahab Uddin and Tuyen Pham Le and TaeChoong Chung and Eui-Nam Huh},
  title = {{Center-Emphasized Visual Saliency and a Contrast-Based Full Reference Image Quality Index}},
  journal = {Symmetry},
  year = {2019},
  doi = {10.3390/sym11030296},
}
Figure