Abstract
Single image reflection removal (SIRR) is a challenging ill-posed problem in computer vision. Traditional deep learning approaches rely on synthetic data or perfectly aligned real-world triplets, both of which have significant drawbacks in capturing real-world complexity or scaling. This paper proposes a weakly-supervised framework that jointly learns to generate and separate reflections using a large-scale unpaired dataset. By employing an entanglement-disentanglement mechanism and cycle consistency, the model achieves state-of-the-art performance on real-world benchmarks while remaining highly efficient.
Introduction
Removing reflections from a single image is a classic problem where a mixture image $M$ must be decomposed into a background layer $B$ and a reflection layer $R$. Existing methods struggle with two main issues: the scarcity of perfectly registered training triplets and the "linear trap," where synthetic data fails to capture the complex physics of real-world reflections, such as non-uniform lighting and ghosting artifacts.
To address these, the authors propose a framework that leverages abundant unpaired data. Instead of relying on simple linear combinations, the model learns the underlying distribution of reflections through a joint generation and separation process, bridging the gap between synthetic training and real-world application.
Methodology
The framework consists of two primary components: a Generator ($G$) and a Separator ($S$), which work together in a cycle-consistent manner.
Joint Generation and Separation
The Generator ($G$) performs "entanglement," learning to fuse a random background and reflection into a realistic mixture. Unlike simple linear addition, $G$ is trained to model the complex physical properties of real reflections. The Separator ($S$) performs "disentanglement," splitting a mixture into its constituent background, reflection, and an auxiliary edge map.
Cycle Consistency and Multi-Task Learning
As shown in the architecture diagram, the model utilizes two cycles to enforce consistency without requiring paired ground truth:
- Generation-Separation Cycle: This cycle starts with separate background ($B$) and reflection ($R$) images. $G$ creates a synthetic mixture $M^*$, which $S$ then attempts to decompose back into $B^*$ and $R^*$. This ensures that the generation process produces mixtures that are actually separable.
- Separation-Generation Cycle: This cycle starts with a real-world mixture $M$. $S$ separates it into $B^*$ and $R^*$, which $G$ then uses to reconstruct the original mixture $M^*$. This ensures the separator captures all relevant information from the real input.
To improve detail preservation, the separator also predicts an edge map $E$. This multi-task approach helps the model distinguish between sharp background features and typically blurrier reflection artifacts. The training is guided by a combination of Adversarial loss (to ensure visual realism), Pixel reconstruction loss ($L_1$), and Content reconstruction loss using VGG feature maps.
Datasets and Experiments
The authors introduced a new Unpaired Reflection Dataset containing 4,027 images of mixtures, backgrounds, and reflections collected across various scenes and lighting conditions. This dataset provides the variety needed for the model to generalize to diverse real-world scenarios.
Evaluation was performed on the standard SIR$^2$ Benchmark. The proposed method achieved a PSNR of 24.48 and an SSIM of 0.903, surpassing previous state-of-the-art models. Furthermore, the model is highly efficient, processing a $224 \times 288$ image in just 0.0214 seconds, making it roughly 15x faster than some competing deep learning approaches.
Conclusion
By treating reflection generation and separation as dual tasks, this framework overcomes the limitations of synthetic and paired data. The use of weak supervision and edge-aware multi-task learning allows for robust, high-quality reflection removal in real-world images. While extreme reflections remain a challenge, the authors suggest that this joint learning paradigm could be extended to other restoration tasks like deraining or dehazing.
Summary updated