TL;DR
This paper introduces "Degraded Reference Image Quality Assessment" (DR IQA), a new paradigm for evaluating image quality when only intermediate, degraded versions of an image are available, not the original pristine one. They develop novel DR IQA models and large-scale datasets, demonstrating significant performance improvements over existing methods in realistic multi-stage distortion scenarios.
What Problem Does This Paper Address?
In real-world media distribution, images and videos often go through multiple stages of quality degradation (e.g., compression, noise, blurring) before reaching the end-user. This makes traditional "full-reference" (FR) or "reduced-reference" (RR) image quality assessment (IQA) methods, which require access to the original pristine image, impractical. While "no-reference" (NR) methods can be used, their performance is often unreliable, especially with multiple distortions. A key challenge is how to effectively utilize intermediate, degraded reference images that are often available (e.g., at a video transcoder input) to improve the quality assessment of the final, multiply-distorted image. This paper addresses this gap by proposing and establishing the DR IQA paradigm.
Methods
The authors propose a new paradigm called Degraded Reference IQA (DR IQA). They define DR IQA architectures using a 6-bit code to represent different configurations of how available information (pristine, degraded reference, final distorted images) is utilized. They focus on two main scenarios:
- Scenario 1 (Type-100100): Assumes the pristine reference image is available early in the pipeline, allowing for calculation of its quality relative to the degraded reference (ASDR). The DR IQA model then predicts the final distorted image's quality (ASFD) using both ASDR and the relative quality between the degraded reference and final distorted image (RSFD).
- Scenario 2 (Type-001100): Assumes the pristine reference is not available. Here, NR IQA methods are used to estimate the quality of the degraded reference image (ASDR), which is then combined with RSFD to predict ASFD.
They develop three DR IQA models:
- Distortion Behavior Model 1: An empirical model based on observed linear relationships between ASFD and RSFD, parameterized by ASDR.
- Distortion Behavior Model 2: A more flexible six-parameter polynomial model to capture non-linear distortion behaviors.
- Support Vector Regression (SVR) Model: A more complex, data-driven model serving as a strong baseline.
Datasets
The paper introduces two new large-scale, synthetically annotated datasets for DR IQA:
- DR IQA database Version 1 (V1)
- DR IQA database Version 2 (V2)
These datasets contain pristine images, singly distorted "degraded reference" (DR) images (Gaussian noise, blur, JPEG compression), and multiply distorted "final distorted" (FD) images resulting from five distortion combinations (Blur-JPEG, Blur-Noise, JPEG-JPEG, Noise-JPEG, Noise-JP2K). They also utilize the existing Waterloo Exploration-II (Exp-II) database for independent testing. All datasets are annotated using a synthetic quality benchmark (SQB) mechanism.
Key Insights and Ideas
- The "intermediate reference" is valuable: The core idea is that even a degraded reference image, when available, provides crucial information that significantly boosts quality assessment accuracy compared to using only NR methods or trying to infer quality from the final image alone.
- Distortion behavior is complex but predictable: The authors observe that the relationship between the quality of the degraded reference (ASDR) and the final distorted image (ASFD), relative to the degraded reference (RSFD), is not always simple. However, they find that these relationships can be modeled, especially by understanding how different distortion combinations interact.
- DR IQA can outperform FR IQA in practice: Surprisingly, their DR IQA models often perform as well as or better than FR IQA methods when tested on datasets where the pristine reference is not available at the final stage, highlighting the limitations of FR IQA in real-world scenarios.
- NR methods can be effectively "boosted": By using NR methods to assess the degraded reference image (Scenario 2), DR IQA can achieve strong performance even without access to the pristine image, demonstrating the utility of NR methods when combined with DR IQA principles.
Key Results
- DR IQA Models 1, 2, and 3 consistently and significantly outperform baseline FR and NR methods across various datasets and distortion combinations.
- The proposed DR IQA models achieve performance comparable to or better than absolute FR IQA scores on test datasets where the pristine reference is not available at the final stage.
- Model 1, the simplest distortion behavior model with only two parameters, often achieves performance similar to the more complex Model 2 and Model 3, underscoring the importance of understanding distortion behavior.
- The newly introduced DR IQA databases V1 and V2 are shown to be effective for training and validating DR IQA models.
Limitations and Future Work
- Distortion Combinations: While the models perform well, certain complex distortion combinations like Noise-JP2K and the "All Data" case still present challenges, indicating room for improvement.
- Model Complexity vs. Performance: The paper notes that the simplest model (Model 1) often performs comparably to more complex models, suggesting that further research could focus on optimizing parameter efficiency.
- Future Research: The authors emphasize that DR IQA is a promising new paradigm and encourage further exploration of different DR IQA architectures and more sophisticated modeling techniques to handle increasingly complex multi-stage distortion scenarios.
Summary updated