Degraded Reference Image Quality Assessment

Shahrukh Athar, Zhou Wang

TL;DR

This paper introduces "Degraded Reference Image Quality Assessment" (DR IQA), a new paradigm for evaluating image quality when only intermediate, degraded versions of an image are available, not the original pristine one. They develop novel DR IQA models and large-scale datasets, demonstrating significant performance improvements over existing methods in realistic multi-stage distortion scenarios.

What Problem Does This Paper Address?

In real-world media distribution, images and videos often go through multiple stages of quality degradation (e.g., compression, noise, blurring) before reaching the end-user. This makes traditional "full-reference" (FR) or "reduced-reference" (RR) image quality assessment (IQA) methods, which require access to the original pristine image, impractical. While "no-reference" (NR) methods can be used, their performance is often unreliable, especially with multiple distortions. A key challenge is how to effectively utilize intermediate, degraded reference images that are often available (e.g., at a video transcoder input) to improve the quality assessment of the final, multiply-distorted image. This paper addresses this gap by proposing and establishing the DR IQA paradigm.

Methods

The authors propose a new paradigm called Degraded Reference IQA (DR IQA). They define DR IQA architectures using a 6-bit code to represent different configurations of how available information (pristine, degraded reference, final distorted images) is utilized. They focus on two main scenarios:

  • Scenario 1 (Type-100100): Assumes the pristine reference image is available early in the pipeline, allowing for calculation of its quality relative to the degraded reference (ASDR). The DR IQA model then predicts the final distorted image's quality (ASFD) using both ASDR and the relative quality between the degraded reference and final distorted image (RSFD).
  • Scenario 2 (Type-001100): Assumes the pristine reference is not available. Here, NR IQA methods are used to estimate the quality of the degraded reference image (ASDR), which is then combined with RSFD to predict ASFD.

They develop three DR IQA models:

  1. Distortion Behavior Model 1: An empirical model based on observed linear relationships between ASFD and RSFD, parameterized by ASDR.
  2. Distortion Behavior Model 2: A more flexible six-parameter polynomial model to capture non-linear distortion behaviors.
  3. Support Vector Regression (SVR) Model: A more complex, data-driven model serving as a strong baseline.

Datasets

The paper introduces two new large-scale, synthetically annotated datasets for DR IQA:

  • DR IQA database Version 1 (V1)
  • DR IQA database Version 2 (V2)

These datasets contain pristine images, singly distorted "degraded reference" (DR) images (Gaussian noise, blur, JPEG compression), and multiply distorted "final distorted" (FD) images resulting from five distortion combinations (Blur-JPEG, Blur-Noise, JPEG-JPEG, Noise-JPEG, Noise-JP2K). They also utilize the existing Waterloo Exploration-II (Exp-II) database for independent testing. All datasets are annotated using a synthetic quality benchmark (SQB) mechanism.

Key Insights and Ideas

  • The "intermediate reference" is valuable: The core idea is that even a degraded reference image, when available, provides crucial information that significantly boosts quality assessment accuracy compared to using only NR methods or trying to infer quality from the final image alone.
  • Distortion behavior is complex but predictable: The authors observe that the relationship between the quality of the degraded reference (ASDR) and the final distorted image (ASFD), relative to the degraded reference (RSFD), is not always simple. However, they find that these relationships can be modeled, especially by understanding how different distortion combinations interact.
  • DR IQA can outperform FR IQA in practice: Surprisingly, their DR IQA models often perform as well as or better than FR IQA methods when tested on datasets where the pristine reference is not available at the final stage, highlighting the limitations of FR IQA in real-world scenarios.
  • NR methods can be effectively "boosted": By using NR methods to assess the degraded reference image (Scenario 2), DR IQA can achieve strong performance even without access to the pristine image, demonstrating the utility of NR methods when combined with DR IQA principles.

Key Results

  • DR IQA Models 1, 2, and 3 consistently and significantly outperform baseline FR and NR methods across various datasets and distortion combinations.
  • The proposed DR IQA models achieve performance comparable to or better than absolute FR IQA scores on test datasets where the pristine reference is not available at the final stage.
  • Model 1, the simplest distortion behavior model with only two parameters, often achieves performance similar to the more complex Model 2 and Model 3, underscoring the importance of understanding distortion behavior.
  • The newly introduced DR IQA databases V1 and V2 are shown to be effective for training and validating DR IQA models.

Limitations and Future Work

  • Distortion Combinations: While the models perform well, certain complex distortion combinations like Noise-JP2K and the "All Data" case still present challenges, indicating room for improvement.
  • Model Complexity vs. Performance: The paper notes that the simplest model (Model 1) often performs comparably to more complex models, suggesting that further research could focus on optimizing parameter efficiency.
  • Future Research: The authors emphasize that DR IQA is a promising new paradigm and encourage further exploration of different DR IQA architectures and more sophisticated modeling techniques to handle increasingly complex multi-stage distortion scenarios.

Summary updated

Abstract

In practical media distribution systems, visual content usually undergoes multiple stages of quality degradation along the delivery chain, but the pristine source content is rarely available at most quality monitoring points along the chain to serve as a reference for quality assessment. As a result, full-reference (FR) and reduced-reference (RR) image quality assessment (IQA) methods are generally infeasible. Although no-reference (NR) methods are readily applicable, their performance is often not reliable. On the other hand, intermediate references of degraded quality are often available, e.g., at the input of video transcoders, but how to make the best use of them in proper ways has not been deeply investigated. Here we make one of the first attempts to establish a new paradigm named degraded-reference IQA (DR IQA). Specifically, by using a two-stage distortion pipeline we lay out the architectures of DR IQA and introduce a 6-bit code to denote the choices of configurations. We construct the first large-scale databases dedicated to DR IQA and have made them publicly available. We make novel observations on distortion behavior in multi-stage distortion pipelines by comprehensively analyzing five multiple distortion combinations. Based on these observations, we develop novel DR IQA models and make extensive comparisons with a series of baseline models derived from top-performing FR and NR models. The results suggest that DR IQA may offer significant performance improvement in multiple distortion environments, thereby establishing DR IQA as a valid IQA paradigm that is worth further exploration.
BibTeX
@misc{athar2021degraded,
  author = {Shahrukh Athar and Zhou Wang},
  title = {{Degraded Reference Image Quality Assessment}},
  year = {2021},
  eprint = {2110.14899},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.14899},
}
Figure