Degraded Reference Image Quality Assessment

Shahrukh Athar, Zhou Wang

TL;DR

This paper introduces Degraded-Reference Image Quality Assessment (DR IQA), a new paradigm for evaluating image quality when the original, pristine image is unavailable, but an intermediate, already-degraded version is accessible. By analyzing how multiple distortions interact, the authors develop simple yet highly effective models that significantly outperform traditional No-Reference methods, opening up new possibilities for real-world media distribution monitoring.

What Problem Does This Paper Address?

In an ideal world, evaluating image quality is straightforward: you compare the distorted image to its pristine original using Full-Reference (FR) metrics like PSNR or SSIM. However, in real-world media delivery networks (like YouTube or Netflix), streaming content undergoes multiple stages of degradation—such as camera noise during capture, followed by initial JPEG compression, and subsequent video transcoding for distribution.

By the time the video reaches an intermediate node or the end-user, the original pristine source is long gone. While No-Reference (NR) or "blind" metrics don't require the original image, they are notoriously unreliable when dealing with multiple, overlapping distortions.

This paper addresses this gap by exploiting a common but underutilized resource: intermediate degraded references (e.g., the video stream right before it enters a transcoder). The authors formalize how to use these imperfect references to accurately predict the final perceived quality.

Methods

The authors formalize the DR IQA problem space by mapping out 64 potential architectures using a 6-bit configuration code based on which inputs (Pristine, Degraded, or Final Distorted images and their respective metrics) are available. They identify 53 valid configurations and deeply explore two practical scenarios:

  1. Scenario 1 (Type-100100): Assumes the absolute quality score of the degraded reference is pre-computed and passed along as side information, which is then combined with the relative FR score between the degraded and final images.
  2. Scenario 2 (Type-001100): A completely passive approach where an NR metric estimates the quality of the degraded reference, which is then combined with the relative FR score.

To predict the final absolute quality score ($AS_{FD}$), the authors design three models:

  • Model 1 (Distortion Behavior Model): A simple, two-parameter linear model derived empirically from observing how quality degrades across stages.
  • Model 2 (Polynomial Model): A six-parameter quadratic model designed to better capture non-linear interactions between different distortion types.
  • Model 3 (SVR Model): A machine learning baseline utilizing Support Vector Regression with a Radial Basis Function (RBF) kernel.

Datasets

To train and validate their models, the authors constructed two new large-scale datasets:

  • DR IQA Database Version 1 (V1) & Version 2 (V2): Each database contains 34 pristine images, generating 1,122 singly-distorted degraded references (blur, JPEG, noise) and 31,790 multiply-distorted final images (combinations like Blur-JPEG, Noise-JPEG, Noise-JP2K).

Because subjective human testing on over 60,000 images is unfeasible, the datasets were annotated using the Synthetic Quality Benchmark (SQB), a highly reliable proxy that fuses multiple top-performing FR metrics.

For independent cross-dataset evaluation, they utilized four existing benchmarks:

  • Waterloo Exploration-II (Exp-II)
  • LIVE Wild Compressed (LIVE WCmp)
  • LIVE Multiply Distorted (LIVE MD)
  • Multiply Distorted IVL (MDIVL)

Key Insights and Ideas

  • The "Overshoot" Phenomenon: Intuitively, adding a second layer of distortion should always make an image look worse. However, the authors discovered that when an image with heavy Gaussian noise undergoes subsequent JPEG or JP2K compression, the final quality score can actually improve. The compression algorithm acts as an unintended low-pass filter, effectively denoising the image.
  • Linearity of Slopes: When plotting the final absolute quality against the relative quality, the relationship for a given first-stage distortion level is remarkably linear. Even more surprising, the slopes ($m$) of these lines scale linearly with the absolute quality of the intermediate reference. This insight allowed the authors to build an incredibly accurate model (Model 1) using just two parameters.
  • NR Metrics as Reliable Proxies: Scenario 2 demonstrates that you don't need to alter network protocols to transmit side information. Off-the-shelf NR metrics (like CORNIA or dipIQ) are accurate enough at evaluating the first stage of distortion to anchor the entire DR IQA pipeline.

Key Results

The proposed DR IQA models were evaluated using Pearson Linear Correlation Coefficient (PLCC) and Spearman Rank-order Correlation Coefficient (SRCC):

  • Outperforming the Baselines: Across all test databases, the DR IQA models comprehensively beat standard No-Reference baselines and simple relative Full-Reference baselines.
  • Beating 2stepQA: Compared to 2stepQA (a previous state-of-the-art method designed specifically for two-stage compression), the proposed models achieved significantly higher correlation scores, particularly on complex, multi-distortion datasets like Waterloo Exp-II and MDIVL.
  • Approaching Absolute FR Performance: In many test cases, the DR IQA models achieved performance parity with absolute Full-Reference metrics that had actual access to the pristine, original images—a remarkable feat given that the DR models operate without it.

Limitations and Future Work

While the models perform exceptionally well on distinct distortion pairs (like Blur followed by JPEG), their performance degrades slightly in the "All Data" scenario, where multiple, highly diverse distortion combinations are mixed together blindly. Capturing the complex, highly non-linear behavior of combinations like Noise-JP2K remains a tough challenge.

Additionally, this study limits its scope to a two-stage distortion pipeline. The authors note that extending this framework to handle $N$-stages of degradation and exploring the remaining 51 valid DR IQA architectures represent promising frontiers for future research.

Summary updated

Abstract

In practical media distribution systems, visual content usually undergoes multiple stages of quality degradation along the delivery chain, but the pristine source content is rarely available at most quality monitoring points along the chain to serve as a reference for quality assessment. As a result, full-reference (FR) and reduced-reference (RR) image quality assessment (IQA) methods are generally infeasible. Although no-reference (NR) methods are readily applicable, their performance is often not reliable. On the other hand, intermediate references of degraded quality are often available, e.g., at the input of video transcoders, but how to make the best use of them in proper ways has not been deeply investigated. Here we make one of the first attempts to establish a new paradigm named degraded-reference IQA (DR IQA). Specifically, we lay out the architectures of DR IQA and introduce a 6-bit code to denote the choices of configurations. We construct the first large-scale databases dedicated to DR IQA and will make them publicly available. We make novel observations on distortion behavior in multi-stage distortion pipelines by comprehensively analyzing five multiple distortion combinations. Based on these observations, we develop novel DR IQA models and make extensive comparisons with a series of baseline models derived from top-performing FR and NR models. The results suggest that DR IQA may offer significant performance improvement in multiple distortion environments, thereby establishing DR IQA as a valid IQA paradigm that is worth further exploration.
BibTeX
@misc{athar2021degraded,
  author = {Shahrukh Athar and Zhou Wang},
  title = {{Degraded Reference Image Quality Assessment}},
  year = {2021},
  eprint = {2110.14899},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.14899},
}
Figure