Abstract
Deep learning models for Single Image Reflection Removal (SIRR) are highly vulnerable to adversarial attacks—tiny, invisible perturbations that can cause restoration to fail completely. This paper presents the first comprehensive robustness study for SIRR and introduces a novel transformer-based architecture designed to defend against these threats. By incorporating an Adversarial Image Discriminator (AID) and multi-scale fusion modules, the proposed model maintains high-quality image restoration even under targeted adversarial attacks, outperforming existing state-of-the-art methods in both clean and corrupted scenarios.
Introduction
Single Image Reflection Removal (SIRR) is a critical task in computer vision, aimed at separating the desired background (transmission) from unwanted glare (reflection) in photos taken through glass. While modern deep learning models have achieved impressive results, they remain remarkably "brittle."
Recent findings reveal that these models are susceptible to adversarial attacks: perturbations so small they are invisible to the human eye but large enough to trick a neural network into producing corrupted, noisy, or entirely incorrect outputs. Prior to this work, there was no standard benchmark to measure the robustness of SIRR models, nor were there specialized architectures designed to defend against targeted attacks in the context of reflection removal.
Methodology
The authors address the robustness gap through a dual approach: systematic attacking and architectural defense.
Systematic Attacking
To evaluate robustness, the researchers adapted the Projected Gradient Descent (PGD) method to create six distinct attack modes for SIRR. These modes vary based on the objective (pixel-wise MSE vs. perceptual LPIPS similarity) and the targeted region (the whole image, reflection areas only, or clean areas only). Their analysis revealed that attacks targeting reflection regions are particularly effective, as networks are already primed to detect and remove distortions in those areas.
The Robust SIRR Architecture
To defend against these attacks, a novel architecture was proposed with three core components:
- Transformer Backbone: The model utilizes "RestoreFormer" blocks. Attention mechanisms have been shown to be inherently more robust than standard convolutions for low-level vision tasks.
- Cross-Scale Attention (CSA) & Multi-Scale Fusion (MSF): These modules process the image at multiple resolutions simultaneously. By narrowing the feature gap between different scales, the model can better distinguish between actual image content and fine-grained adversarial noise.
- Adversarial Image Discriminator (AID): This sub-network determines if an input is "clean" or "adversarial." It then generates dynamic convolution kernels—custom-tailored filters—that allow the main network to adapt its processing based on the perceived noisiness of the input.
Experimental Results
The model was evaluated on three major SIRR benchmarks: Nature (real-world outdoor scenes), SIR² (controlled laboratory settings), and Real (diverse real-world reflections). Training utilized a combination of 7,643 synthetic images and 90 real-world training pairs.
Quantitative Performance
The quantitative improvements were significant, particularly on the Nature dataset. Standard state-of-the-art models proved highly fragile; for instance, the HG21 model saw its performance (measured in PSNR) drop from 20.32 dB on clean images to a disastrous 10.35 dB under a standard MSE attack.
In contrast, the proposed model achieved 20.97 dB on clean images and maintained a robust 18.53 dB even under the same MSE attack. Notably, the architecture outperformed standard models even when those models were given the benefit of adversarial training, suggesting that the structural design (AID and multi-scale fusion) is the primary driver of robustness.
Key Insights
- The Robustness Bottleneck: Typically, training for robustness degrades performance on clean images. The AID module mitigates this by allowing the network to stay sharp on clean images while switching to a defensive mode for corrupted ones.
- Multi-Scale Shielding: Processing images at reduced scales (e.g., 1/2 or 1/4 size) helps the model ignore high-frequency adversarial noise that might overwhelm the network at full resolution.
Conclusion and Future Work
While the proposed model offers superior robustness, it comes with increased computational complexity due to the transformer blocks and multi-scale fusion. Furthermore, the "cat-and-mouse" nature of adversarial machine learning means that new, unseen attack types could still pose a threat. Future research may focus on applying the AID and multi-scale strategies to other fragile low-level vision tasks, such as image deblurring, dehazing, or super-resolution.
Summary updated