Data-Efficient Image Quality Assessment with Attention-Panel Decoder

Guanyi Qin, Runze Hu, Yutao Liu, Xiawu Zheng, Haotian Liu, Xiu Li, Yan Zhang

Introduction

Blind Image Quality Assessment (BIQA) aims to evaluate image quality without a reference, mimicking human perception. While modern approaches leverage Vision Transformers (ViTs) pre-trained on ImageNet, a fundamental gap exists: classification focuses on abstract global semantics, whereas BIQA requires sensitivity to local, low-level degradations. Bridging this gap typically requires massive amounts of human-annotated quality data, which is expensive and labor-intensive to produce.

To address this, we introduce the Data-Efficient Image Quality Transformer (DEIQT). DEIQT achieves state-of-the-art (SOTA) performance while requiring significantly less training data by utilizing a lightweight quality-aware decoder and a novel "attention-panel" mechanism.

Architecture: Bridging the Semantic Gap

DEIQT moves beyond pure-encoder architectures by employing a full Transformer encoder-decoder structure designed to "translate" abstract features into quality-sensitive representations.

The DEIQT architecture overview, featuring a ViT encoder followed by a Quality-Aware Decoder and Attention Panel.

The model consists of three primary components:

  1. ViT Encoder: Extracts multi-level patch embeddings and a global CLS token.
  2. Quality-Aware Decoder: A lightweight, 1-layer decoder that uses the CLS token as a query to cross-attend to patch features. This forces the model to re-examine local regions for quality-degrading artifacts.
  3. Attention-Panel Mechanism: To mimic the Mean Opinion Score (MOS) of human judges, DEIQT initializes $L$ learnable "panel member" embeddings. These are added to the CLS token, allowing each member to learn a unique perspective on quality. The final score is the average of these predictions.

Experimental Results and Data Efficiency

DEIQT was evaluated across eight standard BIQA datasets, including both synthetic (e.g., LIVE, TID2013) and authentic (e.g., KonIQ-10k, SPAQ) distortions.

The model demonstrates remarkable data efficiency. As shown in the figure below, DEIQT outperforms existing SOTA methods on the LIVE dataset even when trained on only 30% of the available data.

Performance comparison on the LIVE dataset showing DEIQT's superiority at various training set ratios.

Key findings include:

  • SOTA Accuracy: DEIQT achieved top-tier Spearman's Rank Correlation Coefficients (SRCC) across all datasets (e.g., 0.980 on LIVE).
  • Efficiency: The model matches SOTA performance using only 30-40% of data for synthetic sets and 60% for complex authentic sets.
  • Fast Convergence: Satisfactory performance is reached in just two training epochs, significantly faster than standard ViT-based models.

Conclusion

DEIQT demonstrates that classification-heavy features can be effectively adapted for regression tasks like BIQA through targeted decoding. By stabilizing gradients and mimicking human subjective panels, the architecture provides a robust, lightweight, and highly data-efficient solution for automated image quality evaluation. This "translation" strategy holds promise for other data-scarce domains in computer vision.

Summary updated

Abstract

Blind Image Quality Assessment (BIQA) is a fundamental task in computer vision, which however remains unresolved due to the complex distortion conditions and diversified image contents. To confront this challenge, we in this paper propose a novel BIQA pipeline based on the Transformer architecture, which achieves an efficient quality-aware feature representation with much fewer data. More specifically, we consider the traditional fine-tuning in BIQA as an interpretation of the pre-trained model. In this way, we further introduce a Transformer decoder to refine the perceptual information of the CLS token from different perspectives. This enables our model to establish the quality-aware feature manifold efficiently while attaining a strong generalization capability. Meanwhile, inspired by the subjective evaluation behaviors of human, we introduce a novel attention panel mechanism, which improves the model performance and reduces the prediction uncertainty simultaneously. The proposed BIQA method maintains a light-weight design with only one layer of the decoder, yet extensive experiments on eight standard BIQA datasets (both synthetic and authentic) demonstrate its superior performance to the state-of-the-art BIQA methods, i.e., achieving the SRCC values of 0.875 (vs. 0.859 in LIVEC) and 0.980 (vs. 0.969 in LIVE). Checkpoints, logs and code will be available at https://github.com/narthchin/DEIQT.
BibTeX
@inproceedings{qin2023data,
  author = {Guanyi Qin and Runze Hu and Yutao Liu and Xiawu Zheng and Haotian Liu and Xiu Li and Yan Zhang},
  title = {{Data-Efficient Image Quality Assessment with Attention-Panel Decoder}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i2.25302},
}
Figure