Abstract
Since human observers are the ultimate receivers of an image, most image quality assessment (IQA) methods are based on analysis of the properties and mechanisms of the human visual system. However, due to the lack of undistorted images for reference, the accuracy of no-reference IQA (NR-IQA) often cannot compete with that of full-reference IQA (FR-IQA). To bridge this performance gap, we propose a NR-IQA method based on a multi-task generative adversarial network (GAN), which attempts to restore dependable hallucinated images to compensate for missing reference images. Two tasks—hallucination images and quality maps—are outputted by the generator and combined with a specific loss to improve reliability. Additionally, two discriminator networks are used to distinguish undistorted images from hallucinated pairs and quality maps from structural similarity index measurement (SSIM) maps. Finally, the hallucinated and distorted images are input into an IQA network to evaluate quality scores. The superiority of our proposed method is verified by experiments on the LIVE, TID2008, and TID2013 datasets.
Introduction
Image Quality Assessment (IQA) is essential for a wide range of digital imaging applications, from social media filters to medical diagnostics. Traditionally, IQA is categorized into Full-Reference (FR-IQA), which compares a distorted image to a perfect original, and No-Reference (NR-IQA), which evaluates an image without any ground truth.
While FR-IQA is highly accurate due to its comparative nature, NR-IQA is far more practical for real-world scenarios where the original image is unavailable. However, NR-IQA typically lags in performance because models struggle to "know" what a perfect version of a specific distorted image should look like. This research aims to bridge that gap by using deep learning to "hallucinate" the missing reference information.
Methodology
The core of the proposed method is a Multi-Task Generative Adversarial Network. Instead of directly regressing a quality score, the system first attempts to reconstruct the missing reference data to enable a comparative evaluation similar to FR-IQA.
The Multi-Task GAN Architecture
The system operates in two main stages, utilizing a complex generator and dual-discriminator setup:
- The Generator (U-Net Architecture): The generator takes a distorted image and performs two tasks simultaneously:
- Hallucination: Reconstructing a clean, undistorted version of the image.
- Quality Mapping: Generating a predicted SSIM map, which acts as an auxiliary guide to help the network understand the spatial distribution of degradation.
- Dual Discriminators: To ensure high fidelity, the authors use two separate discriminator networks ($D_r$ and $D_s$). $D_r$ checks if the hallucinated image looks "real" compared to actual undistorted images, while $D_s$ ensures the generated quality map matches the statistical properties of real SSIM maps.
- IQA-Specific Loss: Standard GANs often struggle with fine textures. To address this, the authors introduced a specific "IQA-loss" based on features from a pre-trained regression network, ensuring the hallucinated image is useful for quality measurement rather than just visually pleasing.
The IQA Network
In the final stage, the distorted image and the hallucinated reference are fed into an IQA network. This network extracts features from both images, fuses them, and regresses a final quality score based on the differences between the two.
Experimental Results
The model was trained and evaluated on three gold-standard IQA datasets: LIVE (779 images), TID2008 (1,700 images), and TID2013 (3,000 images).
Key Insights
- Hallucination as a Bridge: Treating NR-IQA as a "missing data" problem allows the model to leverage the powerful comparative logic of FR-IQA.
- Multi-Tasking for Stability: Forcing the network to predict a quality map (SSIM) alongside the hallucinated image provides a "predetermined direction," making the learning process more robust and stable.
- Performance: The proposed method demonstrated superior performance compared to several state-of-the-art NR-IQA algorithms like HOSA and RankIQA. On the TID2013 dataset, the model achieved a Spearman Rank Correlation Coefficient (SROCC) of 0.914.
Conclusion and Future Work
While the multi-task GAN approach significantly improves NR-IQA accuracy, some limitations remain. In cases of extreme distortion where texture details are completely lost, the GAN may struggle to hallucinate accurate references, leading to a drop in prediction accuracy. Additionally, the computational complexity of GAN ensembles may pose challenges for real-time mobile deployment. Future research will focus on improving the model's generalization to "wild" images containing multiple, overlapping distortion types.
Summary updated