Abstract
Methods for image Super Resolution (SR) have started to benefit from the development of perceptual quality predictors that are designed for super resolved images. However, extensive cross dataset validation studies have not yet been performed on Image Quality Assessment (IQA) for super resolved images. Moreover, powerful natural scene statistics-based approaches for IQA have not yet been studied for SR. To address these issues, we introduced a new dataset of super-resolved images with associated human quality scores. The dataset is based on the existing SupER dataset, which contains real low-resolution images. This new dataset also has 7 SR algorithms at three magnification scales. We selected optimal quality aware features to create two no-reference (NR) opinion-distortion unaware (ODU) IQA models. Using the same set of selected features, we also implemented two NR-IQA opinion/distortion aware (ODA) models. The selection process identified paired-product (PP) features and those derived from discrete cosine transform coefficients (DCT) as the most relevant for the quality prediction of SR images. We conducted cross dataset validation for several state-of-the-art quality algorithms in four datasets, including our new dataset. The conducted experiments indicate that our models achieved better than state-of-the-art performance among the NR-IQA metrics.
Introduction
Super-Resolution (SR) algorithms aim to reconstruct high-quality, high-resolution images from low-resolution inputs. However, evaluating these algorithms is notoriously difficult. Traditional metrics like PSNR and SSIM often fail to capture "perceptual" quality—an image might have a high PSNR but look blurry or full of unnatural artifacts to a human eye. While Full-Reference (FR) metrics exist, they require a "pristine" ground-truth image, which is rarely available in real-world scenarios. This necessitates Blind (No-Reference) IQA, which assesses quality without a reference. Existing blind models often struggle with generalization; they might perform well on the dataset they were trained on but fail when faced with different SR techniques or real-world sensor noise.
Methodology
The proposed approach involves an optimized feature-selection pipeline to identify the most "quality-aware" features for SR images.
- Feature Extraction: A comprehensive set of 306 perceptual features was extracted, including Mean Subtracted Contrast Normalized (MSCN) coefficients, Log-Derivatives, Steerable Pyramids, and Discrete Cosine Transform (DCT) coefficients.
- Optimal Feature Selection: Sequential Forward Floating Search (SFFS) was employed to iteratively select the best combination of feature families that maximize correlation with human scores, preventing overfitting and improving generalization.
- Model Construction:
- ODU (Opinion-Distortion Unaware): These models (Q1 and Q2) measure deviations from "pristine" image statistics without requiring human scores for training.
- ODA (Opinion-Distortion Aware): These models use Random Forest Regression trained on human subjective scores to map features directly to quality predictions.
The SRIJ Dataset and Benchmarking
The study utilizes four major datasets for rigorous cross-dataset validation: MY, QADS, SupER, and the newly introduced Super Resolution Javeriana (SRIJ) Database. The SRIJ dataset consists of 608 images across 32 scenes, built on real-world images from SupER. It includes 7 different SR algorithms at three magnification scales (x2, x3, x4), complete with human subjective scores (MOS).
The selection process revealed that DCT and Paired-Product (PP) features are the most critical. DCT coefficients capture high-frequency details and energy distribution, while PP features capture unnatural spatial dependencies—the "weird" pixel correlations that SR artifacts often introduce.
Results and Discussion
The proposed models achieved state-of-the-art performance among NR-IQA metrics. On the MY dataset, the Q1_ODU model achieved a Pearson Correlation Coefficient (PCC) of 0.847, significantly outperforming standard blind metrics like NIQE (0.48) or IL-NIQE (0.75).
Crucially, the authors found that "more isn't better": a model using all 306 features performed worse in cross-dataset tests than the optimized models (Q1 and Q2) using only 45 and 73 features, respectively. This highlights the importance of pruning irrelevant data to improve model robustness.
Conclusion
While the proposed models show significant improvements, the SupER dataset remains an extreme challenge due to the subtle distortion differences in real-world images. Future work will focus on increasing sensitivity to fine-grained artifacts in deep learning SR models and exploring how these metrics can be integrated directly into the loss functions of SR neural networks to guide them toward more perceptually pleasing results.
Summary updated