Abstract
No-reference image quality assessment (NR-IQA) models often face performance bottlenecks due to over-fitting on limited datasets. While data augmentation is a standard solution in computer vision, traditional methods frequently alter image quality, creating a mismatch between the augmented image and its original quality label. We propose JNDMix, a novel data augmentation technique based on Just Noticeable Difference (JND) noise mixing. By injecting imperceptible noise into training images, JNDMix enriches dataset diversity without compromising label integrity. Our experiments demonstrate that JNDMix significantly enhances the performance, data efficiency, and generalization of state-of-the-art NR-IQA models, including MANIQA, ResNet, and HyperIQA, across major benchmarks like LIVEC and KonIQ-10k.
Introduction
No-reference image quality assessment (NR-IQA) aims to predict human-perceived image quality without a clean reference image. This task is critical for various applications, from social media optimization to medical imaging. However, training robust NR-IQA models is challenging because human subjective scoring—the "ground truth"—is expensive and difficult to scale. Consequently, existing datasets are relatively small, leading to significant over-fitting.
In general computer vision tasks, data augmentation (e.g., cropping, flipping, or adding Gaussian noise) is used to improve model robustness. However, in the context of IQA, these techniques are often counterproductive. Adding visible noise or blurring an image fundamentally changes its quality score. If a high-quality image is augmented with visible distortion but retains its "high quality" label, the model receives contradictory signals, leading to poor convergence and performance degradation.
The JNDMix Methodology
To address the label mismatch problem, we introduce JNDMix. This method leverages the Human Visual System's (HVS) limitations, specifically the Just Noticeable Difference (JND). JND represents the maximum amount of pixel-level change that can occur before a human observer perceives a difference.
The JNDMix workflow consists of four primary steps:
- JND Map Generation: For each training image, a JND map is calculated using a pre-existing model (such as RGB-JND). This map identifies the visibility threshold for every pixel based on local luminance and texture masking.
- Random Noise Scaling: To ensure the noise remains truly imperceptible, the JND map is multiplied by a random scaling factor $\lambda$, where $0 < \lambda \leq 1$.
- Bipolar Injection: A random matrix $r$ (containing values of $1$ or $-1$) is generated. The final noise is the element-wise product of the scaled JND map and this random matrix. This noise is then added to the original image.
- Label Preservation: Because the added noise is mathematically present but visually imperceptible, the original subjective quality score ($y$) remains a valid label for the augmented image ($\tilde{y} = y$).
This approach allows the model to encounter a unique version of the image in every training epoch, effectively regularizing the network without the destructive effects of visible noise.
Experimental Evaluation
We evaluated JNDMix on two prominent NR-IQA benchmarks:
- LIVEC: A dataset featuring 1,162 real-world images with diverse, authentic distortions.
- KonIQ-10k: A larger-scale dataset containing 10,073 images curated from public multimedia sources.
The method was integrated into several architectures, including standard CNNs (ResNet34, ResNet50), specialized IQA networks (DBCNN, HyperIQA), and the Transformer-based MANIQA.
Key Results and Insights
State-of-the-Art Performance
JNDMix consistently improved the performance of all tested models. Notably, when applied to MANIQA, the method achieved state-of-the-art results on both benchmarks, reaching a Spearman Rank Correlation Coefficient (SRCC) of 0.897 on LIVEC and 0.927 on KonIQ-10k.
Boosting Vision Transformers
Vision Transformers (ViTs) are known to be "data-hungry," often underperforming on small datasets. JNDMix proved particularly effective for ViT-based models. For a standard ViT, the SRCC on the LIVEC dataset jumped from 0.776 to 0.815. This suggests that JNDMix provides the necessary diversity to help Transformers learn robust features even when labeled data is scarce.
Data Efficiency and Generalization
In "low-data" experiments—where models were trained on only 10% of the available data—JNDMix provided a significant performance buffer. For instance, HyperIQA saw an SRCC increase of +0.040 on LIVEC compared to the baseline. Furthermore, models trained with JNDMix showed superior generalization in cross-dataset evaluations, indicating that the noise injection helps the model focus on intrinsic quality features rather than dataset-specific artifacts.
Comparison with Standard Noise
A critical finding of our study was the comparison between JNDMix and standard Gaussian noise. While JNDMix improved model accuracy, the addition of Gaussian noise (which is often visible) actually degraded performance. This confirms that preserving the integrity of the quality label is paramount in IQA augmentation.
Conclusion
JNDMix offers a simple yet powerful solution to the data scarcity and label mismatch problems in NR-IQA. By staying "under the radar" of the human visual system, it provides effective regularization that enhances model robustness and data efficiency. As NR-IQA models continue to evolve toward more complex architectures like Transformers, perceptually-aware augmentation techniques like JNDMix will be essential for reaching new levels of performance and generalization.
Summary updated