Robust Single Image Reflection Removal Against Adversarial Attacks

Zhenbo Song, Zhenyuan Zhang, Kaihao Zhang, Wenhan Luo, Zhaoxin Fan, Wenqi Ren, Jianfeng Lu

Abstract

Deep learning models for Single Image Reflection Removal (SIRR) are highly vulnerable to adversarial attacks—tiny, invisible perturbations that can cause restoration to fail completely. This paper presents the first comprehensive robustness study for SIRR and introduces a novel transformer-based architecture designed to defend against these threats. By incorporating an Adversarial Image Discriminator (AID) and multi-scale fusion modules, the proposed model maintains high-quality image restoration even under targeted adversarial attacks, outperforming existing state-of-the-art methods in both clean and corrupted scenarios.

Introduction

Single Image Reflection Removal (SIRR) is a critical task in computer vision, aimed at separating the desired background (transmission) from unwanted glare (reflection) in photos taken through glass. While modern deep learning models have achieved impressive results, they remain remarkably "brittle."

Recent findings reveal that these models are susceptible to adversarial attacks: perturbations so small they are invisible to the human eye but large enough to trick a neural network into producing corrupted, noisy, or entirely incorrect outputs. Prior to this work, there was no standard benchmark to measure the robustness of SIRR models, nor were there specialized architectures designed to defend against targeted attacks in the context of reflection removal.

Methodology

The authors address the robustness gap through a dual approach: systematic attacking and architectural defense.

Systematic Attacking

To evaluate robustness, the researchers adapted the Projected Gradient Descent (PGD) method to create six distinct attack modes for SIRR. These modes vary based on the objective (pixel-wise MSE vs. perceptual LPIPS similarity) and the targeted region (the whole image, reflection areas only, or clean areas only). Their analysis revealed that attacks targeting reflection regions are particularly effective, as networks are already primed to detect and remove distortions in those areas.

The Robust SIRR Architecture

To defend against these attacks, a novel architecture was proposed with three core components:

  1. Transformer Backbone: The model utilizes "RestoreFormer" blocks. Attention mechanisms have been shown to be inherently more robust than standard convolutions for low-level vision tasks.
  2. Cross-Scale Attention (CSA) & Multi-Scale Fusion (MSF): These modules process the image at multiple resolutions simultaneously. By narrowing the feature gap between different scales, the model can better distinguish between actual image content and fine-grained adversarial noise.
  3. Adversarial Image Discriminator (AID): This sub-network determines if an input is "clean" or "adversarial." It then generates dynamic convolution kernels—custom-tailored filters—that allow the main network to adapt its processing based on the perceived noisiness of the input.
The proposed architecture includes a Transformer-based backbone and a novel Adversarial Image Discriminator (AID) to adaptively handle corrupted inputs.

Experimental Results

The model was evaluated on three major SIRR benchmarks: Nature (real-world outdoor scenes), SIR² (controlled laboratory settings), and Real (diverse real-world reflections). Training utilized a combination of 7,643 synthetic images and 90 real-world training pairs.

Quantitative Performance

The quantitative improvements were significant, particularly on the Nature dataset. Standard state-of-the-art models proved highly fragile; for instance, the HG21 model saw its performance (measured in PSNR) drop from 20.32 dB on clean images to a disastrous 10.35 dB under a standard MSE attack.

In contrast, the proposed model achieved 20.97 dB on clean images and maintained a robust 18.53 dB even under the same MSE attack. Notably, the architecture outperformed standard models even when those models were given the benefit of adversarial training, suggesting that the structural design (AID and multi-scale fusion) is the primary driver of robustness.

Performance comparison showing the proposed model's resilience (PSNR) across various attack types compared to a baseline without adversarial defense.

Key Insights

  • The Robustness Bottleneck: Typically, training for robustness degrades performance on clean images. The AID module mitigates this by allowing the network to stay sharp on clean images while switching to a defensive mode for corrupted ones.
  • Multi-Scale Shielding: Processing images at reduced scales (e.g., 1/2 or 1/4 size) helps the model ignore high-frequency adversarial noise that might overwhelm the network at full resolution.

Conclusion and Future Work

While the proposed model offers superior robustness, it comes with increased computational complexity due to the transformer blocks and multi-scale fusion. Furthermore, the "cat-and-mouse" nature of adversarial machine learning means that new, unseen attack types could still pose a threat. Future research may focus on applying the AID and multi-scale strategies to other fragile low-level vision tasks, such as image deblurring, dehazing, or super-resolution.

Summary updated

Abstract

This paper addresses the problem of robust deep single-image reflection removal (SIRR) against adversarial attacks. Current deep learning based SIRR methods have shown significant performance degradation due to unnoticeable distortions and perturbations on input images. For a comprehensive robustness study, we first conduct diverse adversarial attacks specifically for the SIRR problem, i.e. towards different attacking targets and regions. Then we propose a robust SIRR model, which integrates the cross-scale attention module, the multi-scale fusion module, and the adversarial image discriminator. By exploiting the multi-scale mechanism, the model narrows the gap between features from clean and adversarial images. The image discriminator adaptively distinguishes clean or noisy inputs, and thus further gains reliable robustness. Extensive experiments on Nature, SIR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , and Real datasets demonstrate that our model remarkably improves the robustness of SIRR across disparate scenes.
BibTeX
@inproceedings{song2023robust,
  author = {Zhenbo Song and Zhenyuan Zhang and Kaihao Zhang and Wenhan Luo and Zhaoxin Fan and Wenqi Ren and Jianfeng Lu},
  title = {{Robust Single Image Reflection Removal Against Adversarial Attacks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.02365},
}
Figure