Abstract
Recently deep neural networks (DNNs) have achieved significant success in real-world image super-resolution (SR). However, adversarial image samples with quasi-imperceptible noises could threaten deep learning SR models. In this paper, we propose a robust deep learning framework for real-world SR that randomly erases potential adversarial noises in the frequency domain of input images or features. The rationale is that on the SR task clean images or features have a different pattern from the attacked ones in the frequency domain. Observing that existing adversarial attacks usually add high-frequency noises to input images, we introduce a novel random frequency mask module that blocks out high-frequency components possibly containing the harmful perturbations in a stochastic manner. Since the frequency masking may not only destroys the adversarial perturbations but also affects the sharp details in a clean image, we further develop an adversarial sample classifier based on the frequency domain of images to determine if applying the proposed mask module. Based on the above ideas, we devise a novel real-world image SR framework that combines the proposed frequency mask modules and the proposed adversarial classifier with an existing super-resolution backbone network. Experiments show that our proposed method is more insensitive to adversarial attacks and presents more stable SR results than existing models and defenses.
Introduction
Deep neural networks have revolutionized Single Image Super-Resolution (SISR), particularly in "RealSR" tasks where models are trained on actual camera data. However, these models are notoriously fragile. Adversarial attacks—subtle, quasi-imperceptible noises added to an input image—can trick an SR model into producing "structured artifacts" (unnatural patterns and distortions) that severely degrade visual quality. While many defense mechanisms exist for high-level tasks like image classification, they often fail in SR because the goal of SR is to reconstruct fine details. Traditional defenses like heavy blurring often destroy the very high-frequency information the SR model is trying to recover.
Methodology
The proposed defense strategy centers on the frequency domain, specifically using the 2D Discrete Cosine Transform (DCT). The approach consists of two primary components: the Random Frequency Mask (RFM) module and an Adversarial Sample Classifier.
Random Frequency Mask (RFM) Module
Instead of using a fixed low-pass filter, which would blur all images indiscriminately, the RFM stochastically erases high-frequency components. Using a Bernoulli distribution, the module decides whether to zero out specific DCT coefficients. This randomness introduces uncertainty that disrupts the optimization process used by many white-box adversarial attacks and allows the model a chance to reconstruct natural high-frequency textures.
Adversarial Sample Classifier
To prevent the RFM from unnecessarily blurring clean images, a "gatekeeper" classifier analyzes the DCT map of an input image to determine if it has been attacked. If the image is deemed clean, the RFM modules are bypassed. This ensures that the precision of the SR model is maintained for standard inputs while providing robustness against malicious ones.
Experimental Results
The study utilizes the RealSR (DRealSR) dataset, comprising 595 pairs of low-resolution (LR) and high-resolution (HR) images captured via DSLR cameras. This dataset is critical as it captures complex real-world blur kernels and noise patterns.
Quantitative and Qualitative Performance
The proposed method demonstrates a significant leap in stability compared to existing SR models. Under a strong adversarial attack ($\alpha = 8/255$), the proposed model achieved a PSNR of 26.88 dB, a nearly 7.9 dB improvement over the baseline CDC network (18.98 dB).
Visually, while standard models (such as EDSR, RCAN, or RRDB) produce heavy, grid-like artifacts when attacked, the proposed framework maintains clear textures and sharp edges. Furthermore, the RFM and classifier modules are "plug-and-play," showing significant boosts in adversarial robustness when integrated into other architectures like SRResNet.
Conclusion
By leveraging frequency-domain signatures of adversarial noise, this framework provides a robust defense for real-world super-resolution. The combination of stochastic masking and intelligent gating allows for a balance between high-fidelity reconstruction and resilience against structured artifacts. Future work may explore adaptive attacks targeting the classifier itself to further harden the defense.
Summary updated