Abstract
In deep learning for Image Super-Resolution (SR), the prevailing paradigm suggests that larger datasets invariably lead to superior performance. This study challenges that assumption by demonstrating that training on a strategically pruned "core-set" of data can achieve comparable or even better results than training on the full dataset. We propose a loss-value-based sampling method that utilizes a lightweight pre-trained model to rank training samples by difficulty. By prioritizing high-loss "hard" samples and excluding noisy outliers, we show that models trained on only 50% of the data can outperform full-dataset baselines across various architectures, including CNNs and Transformers. Our findings highlight the importance of data quality over raw volume in SR tasks.
Introduction
Image Super-Resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Modern SR models rely on massive datasets like DIV2K to learn complex mappings. However, the computational and storage costs of these datasets are significant. Moreover, not all data samples contribute equally to the learning process; simple, smooth regions (e.g., clear sky) are easily learned and become redundant, while complex textures provide the critical information needed for high-quality reconstruction. This paper explores whether we can prune "easy" fluff from training sets to improve efficiency and performance.
Methodology
We introduce a Loss-Value-based Sampling approach to identify the most informative samples for a "core-set." The methodology involves three primary stages:
- Difficulty Estimation: The entire training dataset is processed through a lightweight, pre-trained proxy model (e.g., FSRCNN).
- Loss Calculation: Mean Squared Error (MSE) is calculated for each sample. High loss values signify "hard" samples with complex textures, while low loss values indicate "easy" samples with smooth gradients.
- Core-Set Selection: Samples are ranked by loss. We focus on Descending Sampling (DES), which selects the hardest samples first.
A key refinement discovered is the "Noise Ceiling": the top 5% of the hardest samples often contain artifacts or misalignments that degrade model performance. Our "Refined" strategy excludes this top 5% and selects the subsequent 50% of the hardest remaining samples.
Experimental Setup
The models were trained using the DIV2K dataset, consisting of 800 high-quality images divided into approximately 32,000 sub-image patches. Evaluation was conducted on five standard benchmarks: Set5, Set14, BSDS100, Urban100, and Manga109. We tested the pruning strategy across multiple architectures, ranging from the simple FSRCNN to the state-of-the-art Transformer-based SwinIR.
Results and Discussion
The experimental results consistently show that training on a pruned dataset is highly effective. As shown in the table below, models trained on the "hardest" 50% of samples (DES) often exceed the performance of the full-dataset baseline.
Key Insights
- Informativeness of Hard Samples: Redundant "easy" samples do not contribute to the model's ability to reconstruct complex details once basic gradients are mastered.
- The Noise Ceiling: Pruning the absolute hardest 5% of samples acts as a data cleaning step, removing outliers that would otherwise confuse the optimization process.
- Model Agnosticism: The benefits of loss-based pruning are observed across different model complexities, suggesting that sample difficulty is an intrinsic property of the data.
- Efficiency: Loss-based pruning outperformed traditional heuristics like Sobel-based edge detection, proving that a proxy model is a more effective judge of sample utility.
Conclusion
This study proves that "more is not always better" in the context of SR training data. By focusing on a 50% core-set of high-quality, high-difficulty samples, we can reduce storage requirements and potentially improve model accuracy. Future work will explore dynamic pruning during training and the integration of these techniques into Neural Architecture Search (NAS) to accelerate the discovery of efficient SR models.
Summary updated