Abstract
Low-level vision tasks, such as denoising and deraining, often suffer from limited dataset diversity and quality. While data augmentation is a standard solution in high-level vision, traditional spatial methods like CutMix often introduce destructive artifacts in low-level tasks. We propose CutFreq, a novel augmentation method that operates in the frequency domain using Discrete Wavelet Transforms (DWT). By "cutting and swapping" specific frequency components between input and ground-truth images, CutFreq helps models bridge the frequency gap and learn fine-grained textures. Our experiments across five low-level tasks demonstrate superior performance, particularly in low-data regimes.
Introduction
Data augmentation is essential for training robust deep learning models. However, in low-level vision, common techniques like Mixup or CutMix can be counterproductive. These methods often create unrealistic boundaries or color blocks that disrupt the local textural consistency required for precise pixel-to-pixel restoration.
A key observation in this work is the persistent "frequency domain gap" between images reconstructed by state-of-the-art models and real ground-truth images. As shown in Figure 1, even when visual results appear satisfactory, their frequency signatures often reveal a failure to capture specific high-frequency details or low-frequency global balances.
Methodology
The proposed CutFreq method shifts augmentation from the spatial domain to the frequency domain using the Discrete Wavelet Transform (DWT). Unlike the Fourier Transform, DWT preserves spatial structure by decomposing an image into four sub-bands: LL (low-frequency approximation), and LH, HL, HH (horizontal, vertical, and diagonal high-frequency details).
The Cut-and-Swap Strategy
The augmentation process involves two distinct stages designed to guide model convergence:
- Stage I (Fast Convergence): The model identifies frequency bands that are particularly difficult to learn for a given task. These bands are swapped from the ground-truth image into the input image, providing "directional" guidance that helps the model focus on missing features early in training.
- Stage II (Performance Approximation): Bands are alternately exchanged to refine the model’s ability to synthesize realistic textures.
Following the swap, an Inverse DWT brings the augmented image back into the spatial domain for standard training.
Frequency Band Normalized $L_1$ Distance (FBND)
To quantify the frequency gap, we introduce the FBND metric. This metric measures how well a model converges on specific frequency bands, allowing for task-specific optimization. For instance, denoising models typically struggle with high-frequency components, while low-light enhancement is more sensitive to low-frequency information.
Experiments and Results
We evaluated CutFreq across five tasks: Smart ISP (RAW to RGB), Image Denoising (SIDD, DND), Image Deraining (Rain100H/L), Image Deblurring (GoPro, HIDE), and Low-light Enhancement (LoL).
Key Findings
- Performance Gains: CutFreq consistently outperformed baselines and other augmentation methods like CutBlur and Mixup. For example, it achieved a +0.17 dB PSNR gain in Smart ISP and improved Restormer's denoising performance on SIDD by 0.06 dB.
- Low-Data Efficiency: CutFreq is remarkably effective when training data is scarce. Models trained with only 5% or 10% of the dataset using CutFreq maintained a significant performance lead over baseline models trained on the same amount of data.
- Qualitative Quality: Visual inspections confirm that CutFreq-trained models produce images with better global contrast and more vivid, realistic details.
Conclusion
CutFreq addresses the limitations of spatial data augmentation in low-level vision by operating in the frequency domain. By leveraging DWT to swap frequency components, it provides a principled way to bridge the gap between generated and real images. Future work may explore more explicit frequency-learning architectures or automated strategies for selecting optimal frequency bands for different tasks.
Summary updated