IE-IQA: Intelligibility Enriched Generalizable No-Reference Image Quality Assessment

Tianshu Song, Leida Li, Hancheng Zhu, Jiansheng QianFrontiers in Neuroscience 2021 · DOI

Abstract

Image quality assessment (IQA) for authentic distortions in the wild is challenging. Though current IQA metrics have achieved decent performance for synthetic distortions, they still cannot be satisfactorily applied to realistic distortions because of the generalization problem. Improving generalization ability is an urgent task to make IQA algorithms serviceable in real-world applications, while relevant research is still rare. Fundamentally, image quality is determined by both distortion degree and intelligibility. However, current IQA metrics mostly focus on the distortion aspect and do not fully investigate the intelligibility, which is crucial for achieving robust quality estimation. Motivated by this, this paper presents a new framework for building highly generalizable image quality model by integrating the intelligibility. We first analyze the relation between intelligibility and image quality. Then we propose a bilateral network to integrate the above two aspects of image quality. During the fusion process, feature selection strategy is further devised to avoid negative transfer. The framework not only catches the conventional distortion features but also integrates intelligibility features properly, based on which a highly generalizable no-reference image quality model is achieved. Extensive experiments are conducted based on five intelligibility tasks, and the results demonstrate that the proposed approach outperforms the state-of-the-art metrics, and the intelligibility task consistently improves metric performance and generalization ability.

Introduction

Most existing Image Quality Assessment (IQA) algorithms excel at spotting "synthetic" distortions—mathematical artifacts like JPEG compression or Gaussian blur added in controlled laboratory environments. However, these models often fail when faced with "authentic" distortions found in real-world scenarios, such as the complex interplay of motion blur, poor lighting, and sensor noise typical of smartphone photography.

The primary challenge is generalization. A model trained on specific laboratory distortions often fails to perform on diverse, "in the wild" datasets. This research posits that image quality is not merely the absence of noise (distortion) but also the presence of usable information, termed intelligibility. If an image is so degraded that its semantic content—such as identifying a subject—is lost, its quality is fundamentally low. Current metrics largely overlook this semantic "understandability" aspect, focusing instead on low-level pixel statistics.

Examples of low-quality images with authentic distortions and low Mean Opinion Scores (MOS).

Methodology

The proposed IE-IQA framework utilizes a bilateral network architecture to bridge the gap between low-level distortion analysis and high-level semantic understanding.

Bilateral Architecture

The network consists of two distinct branches:

  1. Distortion Backbone: A trainable branch based on EfficientNet-B0 that is optimized on image quality datasets to extract conventional distortion features.
  2. Intelligibility Backbone: A frozen branch pre-trained on high-level semantic tasks (e.g., ImageNet classification or MS-COCO object detection). The authors emphasize that freezing this branch is critical; fine-tuning it on relatively small IQA datasets tends to degrade the robust, generalizable semantic features necessary for intelligibility assessment.

Feature Selection and Fusion

To prevent "negative transfer"—where irrelevant semantic data confuses the quality regressor—the authors implemented two feature selection strategies:

  • Contribution-Based Strategy: For classification tasks, the model identifies neural units that contribute most to classification confidence.
  • Sensitivity-Based Strategy: For non-linear tasks, gradient-based derivatives identify features to which the output is most sensitive.

These selected features are integrated via an Intelligibility Enhanced Module to produce a final Mean Opinion Score (MOS).

An image exhibiting motion blur and complex distortions, illustrating the challenge of assessing quality in the wild.

Results and Discussion

The IE-IQA model was evaluated against several state-of-the-art (SOTA) metrics, with a focus on cross-dataset generalization—the most rigorous test for real-world application.

Generalization and Accuracy

When trained on the large-scale KonIQ-10k dataset and tested on the LIVEW (LIVE in the Wild) dataset, IE-IQA achieved a Pearson Linear Correlation Coefficient (PLCC) of 0.839, significantly outperforming deep learning models like DBCNN (0.802) and MetaIQA (0.748). On the CID2013 benchmark, it achieved a Spearman Rank Correlation Coefficient (SRCC) of 0.934.

Efficiency and Semantic Correlation

A key finding is the strong correlation between image quality and AI "confidence." As MOS increases, the top-1 probability of a standard classifier also rises, suggesting that AI, much like humans, struggles to interpret low-quality imagery. Furthermore, IE-IQA is highly efficient; by operating on a single image resize rather than multiple crops, it is approximately 24x faster than high-performing competitors like HyperNet.

Conclusion

The IE-IQA framework demonstrates that integrating intelligibility features significantly enhances the generalization and accuracy of No-Reference Image Quality Assessment. By combining a dedicated distortion backbone with a frozen, feature-selected semantic branch, the model effectively captures what makes an image "good" to both human observers and machine vision systems. Future research will explore more diverse semantic tasks and advanced attention mechanisms to further refine the interaction between distortion and intelligibility.

Summary updated

Abstract

Image quality assessment (IQA) for authentic distortions in the wild is challenging. Though current IQA metrics have achieved decent performance for synthetic distortions, they still cannot be satisfactorily applied to realistic distortions because of the generalization problem. Improving generalization ability is an urgent task to make IQA algorithms serviceable in real-world applications, while relevant research is still rare. Fundamentally, image quality is determined by both distortion degree and intelligibility. However, current IQA metrics mostly focus on the distortion aspect and do not fully investigate the intelligibility, which is crucial for achieving robust quality estimation. Motivated by this, this paper presents a new framework for building highly generalizable image quality model by integrating the intelligibility. We first analyze the relation between intelligibility and image quality. Then we propose a bilateral network to integrate the above two aspects of image quality. During the fusion process, feature selection strategy is further devised to avoid negative transfer. The framework not only catches the conventional distortion features but also integrates intelligibility features properly, based on which a highly generalizable no-reference image quality model is achieved. Extensive experiments are conducted based on five intelligibility tasks, and the results demonstrate that the proposed approach outperforms the state-of-the-art metrics, and the intelligibility task consistently improves metric performance and generalization ability.
BibTeX
@inproceedings{song2021ie,
  author = {Tianshu Song and Leida Li and Hancheng Zhu and Jiansheng Qian},
  title = {{IE-IQA: Intelligibility Enriched Generalizable No-Reference Image Quality Assessment}},
  booktitle = {Frontiers in Neuroscience},
  year = {2021},
  doi = {10.3389/fnins.2021.739138},
}
Figure