Abstract
Training state-of-the-art image restoration models requires massive datasets and significant computational resources. We present TripleD (Distribution-aware Dataset Distillation), a framework that synthesizes a representative "mini-dataset" (1–5% of the original size). TripleD enables training high-performance models on consumer-grade hardware in under 8 hours while maintaining up to 95% of full-dataset performance.
Introduction
Image restoration tasks such as deblurring, denoising, and deraining have seen significant progress driven by increasingly large datasets. However, the computational cost of training on these datasets creates a barrier for researchers without access to high-end GPU clusters. While dataset distillation has succeeded in classification, its application to the pixel-level regression required for restoration has been limited. TripleD addresses this gap by focusing on distribution alignment and informative sample selection.
Methodology
The TripleD pipeline consists of three primary stages:
- Complexity-Based Selection: Using a pre-trained Vision Transformer (ViT-16), images are ranked by entropy. The top 1–2% most informative samples are selected to form a core real subset.
- Latent Distillation (SD3): Instead of pixel-space optimization, TripleD utilizes Stable Diffusion 3 to synthesize high-fidelity training pairs in the latent space, capturing structural information more efficiently.
- Distribution Alignment: A lightweight CNN fine-tunes the synthetic data to ensure its feature-level distribution aligns with the original large-scale dataset using L2 loss and KL divergence.
Key Results and Efficiency
TripleD demonstrates significant efficiency gains across various benchmarks, including deraining, deblurring, and denoising.
- Resource Savings: Training a 4K resolution restoration model on an NVIDIA RTX 3090 takes less than 8 hours, representing a 500% reduction in compute requirements.
- Performance Retention: On the Rain100L dataset, TripleD achieves 32.08 dB PSNR using only 2% of the data, compared to 36.52 dB for the full dataset. As training progresses, the distilled dataset captures the vast majority of the performance of the full dataset.
- Optimal Trade-off: Analysis shows that while performance scales with dataset size, the most significant gains occur within the first 5% of distilled data, with 2% often representing the "Best Trade-off" for practical use.
Discussion and Future Work
TripleD highlights that high-entropy images are more valuable for restoration training. By focusing on the "hard" 2% of the data, the model learns more than it would from a random sample. By distilling into latent space, the framework avoids the noise associated with direct pixel distillation. While a small performance gap remains compared to full-dataset training, TripleD makes state-of-the-art restoration accessible to a wider range of researchers. Future work will explore the relationship between degradation complexity and restoration difficulty to further close the performance gap.
Summary updated