Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild

Weixia Zhang, Kede Ma, Guangtao Zhai, Xiaokang Yang

Abstract

Blind Image Quality Assessment (BIQA) remains a challenging task due to the discrepancy between synthetic laboratory distortions and real-world "wild" images. This paper introduces UNIQUE (Unified No-reference Image Quality and Uncertainty Evaluator), a model that bridges this gap through a pairwise learning-to-rank approach and explicit modeling of human rating uncertainty. By training on diverse datasets simultaneously without score realignment, UNIQUE achieves state-of-the-art generalization and robustness across various distortion scenarios.

Introduction

The goal of BIQA is to predict human-perceived image quality without a reference image. Traditional models face three primary hurdles. First, there is a significant performance drop when models trained on synthetic distortions (e.g., JPEG compression) are applied to real-world images (e.g., sensor noise), and vice versa. Second, different IQA datasets utilize incompatible scoring scales, complicating joint training. Finally, most models treat human quality scores as absolute values, ignoring the inherent variance or "uncertainty" in human opinions, which provides critical information about perceptual ambiguity.

Methodology

The UNIQUE framework addresses these challenges through a unified training strategy and a novel architecture designed to handle diverse data sources.

The UNIQUE framework overview, illustrating the unified training strategy and uncertainty-aware model estimation.

Pairwise Learning-to-Rank

Instead of direct regression to absolute scores, UNIQUE is trained to compare pairs of images. This approach bypasses the need to realign disparate dataset scales. The model predicts the probability that one image is of higher quality than another, allowing it to learn from multiple databases concurrently regardless of their internal scoring ranges.

Uncertainty Modeling and Architecture

The model outputs two values for each image: a quality score ($f_w$) and an uncertainty estimate ($\sigma_w$). It assumes human ratings follow a Gaussian distribution. The architecture utilizes a ResNet-34 backbone, enhanced with second-order bilinear pooling to capture the complex spatial statistics necessary for quality assessment.

Dual Loss Functions

Two loss functions guide the training:

  1. Fidelity Loss: Derived from quantum physics, this measure compares predicted preference probabilities against ground-truth probabilities from human Mean Opinion Scores (MOS). It handles ambiguous pairs more effectively than standard cross-entropy.
  2. Hinge Loss: This acts as a regularizer, ensuring the model's predicted uncertainty aligns with the relative ranking of human rating variances.

Results and Discussion

UNIQUE was evaluated against six major IQA databases, covering both synthetic (LIVE, CSIQ, KADID-10K) and realistic (BID, LIVE Challenge, KonIQ-10K) distortions.

The Arch of Uncertainty

A key finding is the "arch-like" behavior of human rating variance. As shown in the figure below, human opinions are most consistent at the extreme ends of the quality spectrum (very high or very low quality) and most varied for images of mediocre quality. UNIQUE successfully learns to predict this perceptual uncertainty, proving it is a predictable feature of human perception.

The 'Arch' of Uncertainty across six major IQA datasets, showing higher variance in the middle of the quality spectrum.

Performance and Generalization

The model achieved state-of-the-art results, notably reaching a median Spearman Rank-order Correlation Coefficient (SRCC) of 0.896 on the KonIQ-10K dataset. In cross-database evaluations, UNIQUE demonstrated superior robustness compared to existing blind and full-reference models. Furthermore, the training strategy proved to be model-agnostic; applying it to the DB-CNN architecture resulted in performance gains of up to 151% on "wild" datasets.

Conclusion

UNIQUE represents a significant step toward universal blind image quality assessment. By leveraging pairwise ranking and uncertainty modeling, the model overcomes the limitations of dataset-specific training. Future work includes extending this framework to video quality assessment and investigating the limits of mastering diverse distortion types within a single unified model.

Summary updated

Abstract

Performance of blind image quality assessment (BIQA) models has been significantly boosted by end-to-end optimization of feature engineering and quality regression. Nevertheless, due to the distributional shift between images simulated in the laboratory and captured in the wild, models trained on databases with synthetic distortions remain particularly weak at handling realistic distortions (and vice versa). To confront the cross-distortion-scenario challenge, we develop a unified BIQA model and an approach of training it for both synthetic and realistic distortions. We first sample pairs of images from individual IQA databases, and compute a probability that the first image of each pair is of higher quality. We then employ the fidelity loss to optimize a deep neural network for BIQA over a large number of such image pairs. We also explicitly enforce a hinge constraint to regularize uncertainty estimation during optimization. Extensive experiments on six IQA databases show the promise of the learned method in blindly assessing image quality in the laboratory and wild. In addition, we demonstrate the universality of the proposed training strategy by using it to improve existing BIQA models.
BibTeX
@article{zhang2020uncertainty,
  author = {Weixia Zhang and Kede Ma and Guangtao Zhai and Xiaokang Yang},
  title = {{Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild}},
  journal = {IEEE Trans. Image Process.},
  year = {2020},
  doi = {10.1109/tip.2021.3061932},
}

Loading related content...

Figure