Reference-Free Image Quality Metric for Degradation and Reconstruction Artifacts

Han Cui, Alfredo De Goyeneche, Efrat Shimron, Boyuan Ma, Michael LustigarXiv 2024 · DOI · arXiv

Abstract

Standard image quality metrics like PSNR often fail to align with human perception and require a reference image. We present the QF Predictor, a lightweight, reference-free metric trained via self-supervision to predict JPEG Quality Factors. This model generalizes beyond JPEG artifacts to detect Gaussian blur, noise, and MRI undersampling artifacts, offering a robust tool for assessing image quality in real-world scenarios where ground truth is unavailable.

Introduction: The Need for Reference-Free Metrics

In computer vision tasks such as image restoration, deblurring, and super-resolution, evaluating output quality is critical. Traditional metrics like Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) are "reference-based," meaning they require a pristine original image for comparison. However, in many practical applications—such as medical imaging or real-time streaming—a perfect reference is often non-existent.

Furthermore, these metrics frequently fail to capture perceptual quality. As shown in Figure 1, images with similar PSNR values can exhibit different visual characteristics, and a higher PSNR score does not always equate to a "better" looking image to a human observer. This necessitates a metric that can discern quality based on learned prior knowledge of "clean" versus "degraded" visual patterns.

Figure 1: Comparison of PSNR values for different image degradations. Note how PSNR may not perfectly correlate with perceived visual quality.

Methodology: Learning from JPEG Artifacts

The proposed QF Predictor is a seven-layer fully convolutional network (FCN) designed to assess image quality without a reference. The core innovation lies in its self-supervised training strategy.

Training Strategy

The model is trained to predict the JPEG Quality Factor (QF) of image patches. By taking high-quality images and applying random JPEG compression levels (ranging from 0 to 100), the researchers created a massive, automatically labeled dataset.

Figure 2: Visual representation of varying JPEG Quality Factors (QF). The QF Predictor learns to identify these artifacts to estimate overall image quality.

Key technical insights include:

  • Regression Approach: Treating QF prediction as a continuous regression task rather than classification improved sensitivity to subtle artifacts.
  • Logarithmic Sampling: Training on a distribution that favors high-quality images (QF 80–100) forced the network to learn the "indistinct imperfections" that characterize near-perfect images, rather than just focusing on obvious blocking artifacts found in low-quality samples.

Generalization and Results

Despite being trained only on JPEG artifacts, the QF Predictor generalizes remarkably well to other types of distortion. Because the network learns to recognize "unnatural" pixel patterns, it views Gaussian blur or "Salt & Pepper" noise as a reduction in the effective Quality Factor.

  • Cross-Distortion Sensitivity: The model's predicted QF decreases linearly with increasing levels of Gaussian blur or noise, indicating it has learned a general representation of image "naturalness."
  • MRI Assessment: In medical imaging (using the FastMRI dataset), the model successfully predicted undersampling rates in MRI reconstructions. This provides a reference-free way to flag scans where acceleration rates have compromised diagnostic quality.
  • Dataset Benchmarking: The metric identified that benchmark datasets like LIVE have lower average quality ($0.78 \pm 0.05$) than high-resolution repositories like Flickr1024 ($0.95 \pm 0.04$).

Conclusion and Future Directions

The QF Predictor offers a lightweight and effective way to measure image quality in the absence of a reference. While it shows promise as a perceptual loss function for training restoration networks, care must be taken to balance it with data-consistency terms. Over-reliance on the metric can lead to "generative artifacts"—hallucinated patterns that improve visual "look" at the expense of semantic accuracy. Future research will focus on optimizing this balance for specific domains like medical diagnostics.

Summary updated

Abstract

Image Quality Assessment (IQA) is essential in various Computer Vision tasks such as image deblurring and super-resolution. However, most IQA methods require reference images, which are not always available. While there are some reference-free IQA metrics, they have limitations in simulating human perception and discerning subtle image quality variations. We hypothesize that the JPEG quality factor is representatives of image quality measurement, and a well-trained neural network can learn to accurately evaluate image quality without requiring a clean reference, as it can recognize image degradation artifacts based on prior knowledge. Thus, we developed a reference-free quality evaluation network, dubbed "Quality Factor (QF) Predictor", which does not require any reference. Our QF Predictor is a lightweight, fully convolutional network comprising seven layers. The model is trained in a self-supervised manner: it receives JPEG compressed image patch with a random QF as input, is trained to accurately predict the corresponding QF. We demonstrate the versatility of the model by applying it to various tasks. First, our QF Predictor can generalize to measure the severity of various image artifacts, such as Gaussian Blur and Gaussian noise. Second, we show that the QF Predictor can be trained to predict the undersampling rate of images reconstructed from Magnetic Resonance Imaging (MRI) data.
BibTeX
@misc{cui2024reference,
  author = {Han Cui and Alfredo De Goyeneche and Efrat Shimron and Boyuan Ma and Michael Lustig},
  title = {{Reference-Free Image Quality Metric for Degradation and Reconstruction Artifacts}},
  year = {2024},
  eprint = {2405.02208},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.02208},
}
Figure