Abstract
Although Convolutional Neural Networks (CNNs) have high accuracy in image recognition, they are vulnerable to adversarial examples and out-of-distribution data, and the difference from human recognition has been pointed out. In order to improve the robustness against out-of-distribution data, we present a frequency-based data augmentation technique that replaces the frequency components with other images of the same class. When the training data are CIFAR10 and the out-of-distribution data are SVHN, the Area Under Receiver Operating Characteristic (AUROC) curve of the model trained with the proposed method increases from 89.22% to 98.15%, and further increased to 98.59% when combined with another data augmentation method. Furthermore, we experimentally demonstrate that the robust model for out-of-distribution data uses a lot of high-frequency components of the image.
Introduction
Convolutional Neural Networks (CNNs) have achieved remarkable success in image classification tasks. However, they remain "brittle" when encountering out-of-distribution (OOD) data—images that fall outside the distribution of the training set. In such cases, CNNs often issue high-confidence but incorrect predictions. While standard data augmentation techniques like Mixup or CutMix improve general test accuracy, they can paradoxically degrade OOD detection performance. This research addresses the perceptual gap between humans, who are sensitive to structural "phase" information, and CNNs, which are often overly reliant on frequency "amplitude."
Methodology: Replacement of Frequency Component (RFC)
The proposed Replacement of Frequency Component (RFC) method is a frequency-domain data augmentation technique designed to force models to learn more robust features. Unlike methods that blend different classes, RFC operates exclusively on images within the same class to preserve semantic integrity.
The RFC process involves five key steps:
- Decomposition: Two distinct images from the same class (e.g., two "ship" images) are selected.
- Fourier Transform: Both images are converted from the spatial domain to the frequency domain using a Discrete Fourier Transform (DFT).
- Filtering: Mask matrices separate the frequency spectra into low-frequency components (representing global structure and color) and high-frequency components (representing edges and fine details).
- Component Swapping: A new augmented frequency representation is created by combining the low frequencies of the first image with the high frequencies of the second.
- Reconstruction: An Inverse DFT (IDFT) transforms the mixed frequency data back into the spatial domain for training.
Experimental Results
The authors evaluated RFC using CIFAR10 as the training set and several datasets (SVHN, LSUN, ImageNet, and CIFAR100) for OOD evaluation.
Out-of-Distribution Detection
The RFC method significantly outperformed the baseline and standard augmentations in OOD detection. On the SVHN dataset, the baseline AUROC of 89.22% was boosted to 98.15% by RFC. When combined with Amplitude-Phase Recombination (APR), the performance reached a near-perfect 98.59%.
Robustness to Corruptions
On the CIFAR10-C benchmark, which tests robustness against 19 types of image corruptions (e.g., noise, blur, fog), the RFC+APR combination achieved an average accuracy of 75.86%, compared to the baseline's 57.52%. Notably, while Mixup achieved the highest standard test accuracy (95.31%), its OOD performance was inferior to the baseline, highlighting the trade-off between raw accuracy and robustness.
Discussion: The Role of High Frequencies
A significant insight from this work is that models robust to OOD data utilize high-frequency components more effectively. While standard models often treat high-frequency information as negligible noise, RFC-trained models leverage these details to better distinguish between classes and detect anomalies.
Furthermore, RFC is a "plug-and-play" augmentation. Its synergy with APR suggests that frequency distribution and phase information provide complementary pathways to robustness. However, limitations remain; RFC is less effective against Gaussian blur, which acts as a low-pass filter and removes the high-frequency cues the model has learned to rely on.
Conclusion
Replacement of Frequency Component (RFC) offers a powerful way to enhance CNN robustness without sacrificing significant accuracy. By focusing on same-class frequency swapping, the method bridges the gap between human and machine perception, leading to state-of-the-art OOD detection performance.
Summary updated