Natural DCT statistics approach to no-reference image quality assessment

Michele A. Saad, Alan C. Bovik, Christophe Charrier

TL;DR

This paper introduces a computationally efficient no-reference image quality assessment (NR-IQA) method that uses only three features extracted from the Discrete Cosine Transform (DCT) domain. By leveraging these DCT statistics, the approach achieves high correlation with human visual perception, offering a promising alternative to more complex NR-IQA methods.

What Problem Does This Paper Address?

Evaluating image quality without a pristine reference image (no-reference or blind image quality assessment) is a significant challenge. Existing methods often fall into two categories: those that are specific to certain types of distortions (like blur or blockiness) and thus limited in scope, or those that rely on complex machine learning models trained on a large number of features. The latter can be computationally expensive and their performance is heavily dependent on the representativeness of the chosen features. This paper aims to bridge this gap by developing a general-purpose NR-IQA method that is both effective and computationally efficient, correlating well with human judgment.

Methods

The core of this approach lies in extracting features from the DCT domain of image patches. The authors hypothesize that natural images have specific statistical properties in the DCT domain, and that distortions alter these statistics. They focus on three specific DCT-based features:

  • Kurtosis: This feature quantifies the "peakedness" and "tail weight" of the DCT coefficient histogram. Distorted images tend to exhibit different kurtosis values compared to natural images.
  • Anisotropy: This feature captures how directional information in an image is affected by distortions. It's computed by analyzing DCT coefficients from image patches oriented at different angles (0°, 45°, 90°, 135°). Degradations are expected to reduce this directional dependency.
  • Entropy Variance: While not explicitly detailed in the provided text, the paper mentions using "entropy variance" as a feature. This likely relates to the distribution of DCT coefficients and how it varies across different image regions.

These three features are extracted from 17x17 DCT patches. To improve robustness, the authors also apply these feature extraction steps at two different scales: the original image and a down-sampled version. A Support Vector Machine (SVM) with a third-order polynomial kernel is then trained using these multi-scale features to predict the image quality score.

Datasets

The primary dataset used for training and evaluation is the LIVE IQA database [11]. This database contains images with various types of distortions, including JPEG2000, JPEG, white noise, Gaussian blur, and fast fading channel distortions. The authors split this database into training and testing sets, ensuring that images from the same original scene are not present in both sets to avoid content bias.

Key Insights and Ideas

  • DCT Domain as a Rich Source of Quality Information: The paper demonstrates that simple statistical measures of DCT coefficients can effectively capture image quality degradation, even without a reference image.
  • Minimal Feature Set for High Performance: The success of using just three DCT-based features highlights that complex feature engineering might not always be necessary for effective NR-IQA. This significantly reduces computational burden.
  • Hybrid Approach: The method combines elements of "natural scene statistics" (NSS) approaches (by leveraging inherent image statistics) and "learning-based" approaches (by using an SVM for prediction).
  • Multi-scale Analysis Improves Robustness: Applying the feature extraction at different image resolutions helps to capture quality information that might be scale-dependent, leading to better overall performance.

Key Results

The paper reports strong correlations between the predicted quality scores from their method and subjective Mean Opinion Scores (MOS) from the LIVE IQA database.

  • Spearman Correlation: The highest individual feature correlations with subjective DMOS were observed for Kurtosis (up to 0.9481) and Max Entropy (up to 0.9617) across different distortion types.
  • Overall Performance: When combining the three features and using the SVM model, the method achieved high Spearman correlation coefficients across various subsets of the LIVE database, with values ranging from 0.8134 (Fast Fading) to 0.9747 (White Noise). The overall correlation for the entire database was 0.8248.
  • Computational Efficiency: The approach is described as "computationally convenient" due to the small number of features extracted entirely within the DCT domain.

Limitations and Future Work

  • Limitations: While the paper demonstrates good performance, the authors acknowledge that some individual feature correlations can be low for certain distortion types (e.g., Fast Fading). The performance might also be sensitive to the specific choice of DCT patch size (17x17) and the normalization method used.
  • Future Work: The authors suggest that further exploration of multi-scale analysis and potentially incorporating more features from the DCT domain could lead to even more robust and accurate NR-IQA systems. They also imply that this approach could be extended to video quality assessment.

Summary updated

Abstract

General-purpose no-reference image quality assessment approaches still lag the advances in full-reference methods. Most no-reference methods are either distortion specific (i.e. they quantify one or more distortions such as blur, blockiness, or ringing), or they train a learning machine based on a large number of features. In this approach, we propose a discrete cosine transform (DCT) statistics-based support vector machine (SVM) approach based on only 3 features in the DCT domain. The approach extracts a very small number of features and is entirely in the DCT domain, making it computationally convenient. The results are shown to correlate highly with human visual perception of quality.
BibTeX
@inproceedings{saad2010natural,
  author = {Michele A. Saad and Alan C. Bovik and Christophe Charrier},
  title = {{Natural DCT statistics approach to no-reference image quality assessment}},
  booktitle = {ICIP},
  year = {2010},
  url = {https://openreview.net/forum?id=fKsUuoyzEHN},
}
Figure