Proceedings of the 29th ACM International Conference on Multimedia

Jiutao Yue, Haofeng Li, Pengxu Wei, Guanbin Li, Liang LinOpenalex Unknown 2021 · DOI · arXiv · OpenAlex

Abstract

Recently deep neural networks (DNNs) have achieved significant success in real-world image super-resolution (SR). However, adversarial image samples with quasi-imperceptible noises could threaten deep learning SR models. In this paper, we propose a robust deep learning framework for real-world SR that randomly erases potential adversarial noises in the frequency domain of input images or features. The rationale is that on the SR task clean images or features have a different pattern from the attacked ones in the frequency domain. Observing that existing adversarial attacks usually add high-frequency noises to input images, we introduce a novel random frequency mask module that blocks out high-frequency components possibly containing the harmful perturbations in a stochastic manner. Since the frequency masking may not only destroys the adversarial perturbations but also affects the sharp details in a clean image, we further develop an adversarial sample classifier based on the frequency domain of images to determine if applying the proposed mask module. Based on the above ideas, we devise a novel real-world image SR framework that combines the proposed frequency mask modules and the proposed adversarial classifier with an existing super-resolution backbone network. Experiments show that our proposed method is more insensitive to adversarial attacks and presents more stable SR results than existing models and defenses.

Introduction

Deep neural networks have revolutionized Single Image Super-Resolution (SISR), particularly in "RealSR" tasks where models are trained on actual camera data. However, these models are notoriously fragile. Adversarial attacks—subtle, quasi-imperceptible noises added to an input image—can trick an SR model into producing "structured artifacts" (unnatural patterns and distortions) that severely degrade visual quality. While many defense mechanisms exist for high-level tasks like image classification, they often fail in SR because the goal of SR is to reconstruct fine details. Traditional defenses like heavy blurring often destroy the very high-frequency information the SR model is trying to recover.

Methodology

The proposed defense strategy centers on the frequency domain, specifically using the 2D Discrete Cosine Transform (DCT). The approach consists of two primary components: the Random Frequency Mask (RFM) module and an Adversarial Sample Classifier.

Random Frequency Mask (RFM) Module

Instead of using a fixed low-pass filter, which would blur all images indiscriminately, the RFM stochastically erases high-frequency components. Using a Bernoulli distribution, the module decides whether to zero out specific DCT coefficients. This randomness introduces uncertainty that disrupts the optimization process used by many white-box adversarial attacks and allows the model a chance to reconstruct natural high-frequency textures.

The Random Frequency Mask (RFM) module pipeline: input features are transformed via DCT, stochastically masked, and then reconstructed via Inverse-DCT.

Adversarial Sample Classifier

To prevent the RFM from unnecessarily blurring clean images, a "gatekeeper" classifier analyzes the DCT map of an input image to determine if it has been attacked. If the image is deemed clean, the RFM modules are bypassed. This ensures that the precision of the SR model is maintained for standard inputs while providing robustness against malicious ones.

Experimental Results

The study utilizes the RealSR (DRealSR) dataset, comprising 595 pairs of low-resolution (LR) and high-resolution (HR) images captured via DSLR cameras. This dataset is critical as it captures complex real-world blur kernels and noise patterns.

Quantitative and Qualitative Performance

The proposed method demonstrates a significant leap in stability compared to existing SR models. Under a strong adversarial attack ($\alpha = 8/255$), the proposed model achieved a PSNR of 26.88 dB, a nearly 7.9 dB improvement over the baseline CDC network (18.98 dB).

Qualitative comparison of super-resolution results under adversarial attack. While standard models like EDSR and RCAN produce heavy grid-like artifacts, the proposed framework maintains clear textures and sharp edges.

Visually, while standard models (such as EDSR, RCAN, or RRDB) produce heavy, grid-like artifacts when attacked, the proposed framework maintains clear textures and sharp edges. Furthermore, the RFM and classifier modules are "plug-and-play," showing significant boosts in adversarial robustness when integrated into other architectures like SRResNet.

Conclusion

By leveraging frequency-domain signatures of adversarial noise, this framework provides a robust defense for real-world super-resolution. The combination of stochastic masking and intelligent gating allows for a balance between high-fidelity reconstruction and resilience against structured artifacts. Future work may explore adaptive attacks targeting the classifier itself to further harden the defense.

Summary updated

Abstract

Recently deep neural networks (DNNs) have achieved significant success in real-world image super-resolution (SR). However, adversarial image samples with quasi-imperceptible noises could threaten deep learning SR models. In this paper, we propose a robust deep learning framework for real-world SR that randomly erases potential adversarial noises in the frequency domain of input images or features. The rationale is that on the SR task clean images or features have a different pattern from the attacked ones in the frequency domain. Observing that existing adversarial attacks usually add high-frequency noises to input images, we introduce a novel random frequency mask module that blocks out high-frequency components possibly containing the harmful perturbations in a stochastic manner. Since the frequency masking may not only destroys the adversarial perturbations but also affects the sharp details in a clean image, we further develop an adversarial sample classifier based on the frequency domain of images to determine if applying the proposed mask module. Based on the above ideas, we devise a novel real-world image SR framework that combines the proposed frequency mask modules and the proposed adversarial classifier with an existing super-resolution backbone network. Experiments show that our proposed method is more insensitive to adversarial attacks and presents more stable SR results than existing models and defenses.
BibTeX
@inproceedings{yue2021proceedings,
  author = {Jiutao Yue and Haofeng Li and Pengxu Wei and Guanbin Li and Liang Lin},
  title = {{Proceedings of the 29th ACM International Conference on Multimedia}},
  booktitle = {Openalex Unknown},
  year = {2021},
  doi = {10.1145/3474085},
}
Figure